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TRANSMITTAL LETTER 
TO THE UNITED STATES ELECTED OFFICE (EO/US) 

(ENTRY INTO U.S. NATIONAL PHASE UNDER CHAPTER II) 

PCT/IL00/00434 23 JULY 2000 25 JULY 1999 

INTERNATIONAL APPLICATION NO. INTERNATIONAL FILING DATE PRIORITY DATE CLAIMED 

OPTICAL INSPECTION SYSTEM 

TITLE OF INVENTION 

1. Uri GOLD; 2. Eli PARENTE; 3. Tally GIL AT- B ERN S HTE1N ; 4. Edward BARANOVSKY 

5. Tamir MARGALIT. 

APPLICANT(S) 



Box PCT 

Assistant Commissioner for Patents 
Washington D.C. 20231 

ATTENTION: EO/US 



CERTIFICATION UNDER 37 C.F.R. 1.10* 

(Express Mail label number is mandatory) 
(Express Mail certification is optional) 

I hereby certify that this correspondence and the documents referred to as attached therein are being dispt\sited with the 

United States Postal Service on this date JANUARY 14. 2002 , in an envelope as 'Ti«fress/Mail Post Office tc 

Addressee," Mailing Label Number EV01 1020142US , addressed to the- Assistant Commissioner for 

Patents, Washington, D.C. 2023 1 . 

CONNIE YANNt^TTj/ 
(type />• prjnt name Jf pe/son ma)ttng paper) 

( f^^/O^H^t • 

Signature of pe/son mailing paper 

WARNING: Certificate of mailing (first class) or facsimile transmission procedures of 3 7CFR 18 cannot be used to 

obtain a date of mailing or transmission for this correspondence. 

* WARNING: Each paper or fee filed by "Express Mail" must have the number of the "Express Mail" mailing label 
placed thereon prior to mailing 37 C F R 1 10(b). 

"Since the filing of correspondence under § 1.10 without the Express Mail mailing label thereon is an 
oversight that can be avoided by the exercise of reasonable care, requests for waiver of this requirement 
will not be granted on petition. " Notice of Oct. 24, 1996, 60 Fed Reg 56,439, at 56.442. 
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NOTE The completion of those filing requirements that can be made at a time later than 30 months from the priority date 
results from the Commissioner exercising his judgment under the authority granted under 35 {JSC 37 1(d) The filing 
receipt will show the actual date of receipt of the last item completing the entry into the national phase. See 37 C F R 
§1 491 which states- "An international application enters the national state when the applicant has filed the 
documents and fees required by 35 USC 371(c) within the periods set forth in § 1 494 and § I 495. " 

WARNING: Where the items are those which can be submitted to complete the entry of the international application into 

the national phase are subsequent to 30 months from the priority date the application is still considered to 
be in the international state and if mailing procedures are utilized to obtain a date the express mail 
procedure of37CFR §1.10 must be used (since international application papers are not covered by an 
ordinary certificate of mailing - See 37 C F.R §18 

NOTE Documents and fees must be clearly identified as a submission to enter the national state under 35 USC 3 71 otherwise 
the submission will be considered as being made under 35 USC 111 37 C.F.R. § 1 .494(f). 



1. Applicant herewith submits to the United States Elected Office (EO/US) the following items 
under 35 U.S.C. 371: 

a. [X] This express request to immediately begin national examination procedures (35 

U.S.C. 371(f)). 

b. [X] The U.S. National Fee (35 U.S.C. 371(c)(1)) and other fees (37 C.F.R. § 1.492) 

as indicated below: 
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2. Fees 



CLAIMS FEE 
BASIC FEE** 

SMALL 
ENTITY 


(l)FOR 


(2) NUMBER 
FILED 


(3) NUMBER 
EXTRA 


(4) RATE 


(5) CALCULATIONS 


TOTAL CLAIMS* 


224- 20 = 


204 


x $ 18.00 = 


$NOTPAID 


INDEPENDENT 
CLAIMS* 


52-3 = 


49 


x $84 00 = 


NOT PAID 


.MULTIPLE DEPENDENT CLAIM(S) (if applicable) + $280.00 




[ ] U.S. PTO WAS INTERNATIONAL PRELIMINARY EXAMINATION 
AUTHORITY 

Where an International preliminary examination fee as set forth in § 1.482 has been 
paid on the international application to the U.S. PTO: 

[ ] and the international preliminary examination report states that the 
criteria of novelty, inventive step (non-obviousness) and industrial 
activity, as defined in PCT Article 33(2) to (4) have been satisfied for all 
the claims presented in the application entering the national stage (37 
CFR 1 492(a)(4)) $100 00 

[ ] and the above requirements are not met (37 CFR 1 .492(a)( 1 )) 

[ X ] U.S. PTO WAS NOT INTERNATIONAL PRELIMINARY EXAMINATION 
AUTHORITY 

Where no international preliminary examination fee as set forth 
in § 1.482 has been paid to the U.S. PTO, and payment of an 
international search fee as set forth in § 1.445(a)(2) to the U.S. 
PTO: 

[ ] has been paid (37 CFR 1 .492(a)(2)) $740.00 

[X ] has not been paid (37 CFR 1.492(a)(3)) $1,040.00 

| j where a search report on [he international application has been prepared 

by the European Patent Office or the Japanese Patent Office (37 CFR 

1.492(a)(5)) $890 00 




Total of above Calculations 


=1040.00 


Reduction by l A for filing by small entity, if applicable. Statement may also be filed, (note 37 
CFR 1.9, 1.27, 1.28}_ 




Subtotal 


1040.00 


Total National Fee 


$1040.00 


Fee for recording the enclosed assignment document $40.00 (37 CFR 1 .21(h)). (See Item 13 
below). See attached "ASSIGNMENT COVER SHEET". 




TOTAL 


Total Fees enclosed 


$1040.00 



*May include Preliminary Amendment (see page 8) reducing the number of claims. 
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i. [ X ] A check in the amount of $ 1040.00 to cover the above fees is enclosed. 

ii. [ ] Please charge Account No. in the amount of $ • 

A duplicate copy of this sheet is enclosed. 

'*WARNING. "To avoid abandonment of the application the applicant shall furnish to the United States Patent and 

T rademark Office not later than the expiration of 30 months from the priority date ■ * * * (2) the basic 
national fee (see § 1.492(a)). The 30-month time limit may not be extended. " 37 C.F R § 1.495(b). 

VARNING If the translation of the international application and/or the oath or declaration have not been 

submitted by the applicant within thirty (30) months from the priority date, such requirements may be 
met within a time period set by the Office 37 C F R. § 1. 495(b)(2) The payment of the surcharge set 
forth in § 1.492(e) is required as a condition for accepting the oath or declaration later than thirty (30) 
months after the priority date. The payment of the processing fee set forth in § 1 492(f) is required for 
acceptance of an English translation later than thirty (30) months after the priority date. Failure to 
comply with these requirements will result in abandonment of the application. The provisions of § 1. 136 
apply to the period which is set. Notice of Jan 3, 1993, 1147 OG 29 to 40. 



[ ] Applicant hereby asserts status as a small entity under 37 C.F.R. § 1 .27. 
[ ] A Statement or Written Assertion is attached. 



NOTE 37 CF.R § 1 27(c) deals with the assertion of small entity status, whether by a written specific declaration thereof 
or by payment as a small entity of the basic filing fee or the fee for the entry into the national phase as states. 

"(c) Assertion of small entity status Any party (person, small business concern or nonprofit organization) 
should make a determination, pursuant to paragraph (f) of this section, of entitlement to be accorded small 
entity status based on the definitions set forth in paragraph (a) of this section, and must, in order to 
establish small entity status for the purpose of paying small entity fees, actually make an assertion of 
entitlement to small entity status, in the manner set forth in paragraph (c)(1) or (c)(3) of this section, in the 
application or patent in which such small entity fees are to be paid 

(1 ) Assertion by writing. Small entity status may be established by a written assertion of entitlement 

to small entity status A written assertion must 



(i) Be clearly identifiable, 

(ii) Be signed (see paragraph (c)(2) of this section); and 

(lii) Convey the concept of entitlement to small entity status, such as by stating that 
applicant is a small entity, or that small entity status is entitled to be asserted for the 
application or patent. While no specific words or wording are required to assert small 
entity status, the intent to assert small entity status must be clearly indicated in order 
to comply with the assertion requirement. 

(2) Parties who can sign and file the written assertion. The written assertion can be signed by. 

(i) One of the parties identified in §§ 1.33(b) (e.g., an attorney or agent registered with the 

Office), §§ 3 73(b) of this chapter notwithstanding, who can also file the written 
assertion, 



(u) At least one of the individuals identified as an inventor (even though a §§ 1.63 executed 

oath or declaration has not been submitted), notwithstanding §§ 1.33(b)(4), who can 
also file the written assertion pursuant to the exception under §§ 1 33(b) of this part; 



(Hi) An assignee of an undivided part interest, notwithstanding §§ 1 33(b)(3) and 3.773(b) 
of this chapter, but the partial assignee cannot file the assertion without resort to a 
party identified under §§ 1 33(b) of this part. 
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(3) Assertion by payment of the small entity basic filing or basic national fee The payment, by any 

party, of the exact amount of one of the small entity basic filing fees set forth in §§ 1. 1 6(a), (f), 
(g), (h), or (k), or one of the small entity basic national fees set forth in §§ 1 492(a)(1), (a)(2), 
(a)(3), (a)(4) or (a)(5), will be treated as a written assertion of entitlement to small entity status 
even if the type of basic filing or basic national fee is inadvertently selected in error. 

(i) If the Office accords small entity status based on payment of a small entity basic filing 

or basic national fee under paragraph (c)(3) of this section that is not applicable to that 
application, any balance of the small entity fee that is applicable to that application 
will be due along with the appropriate surcharge set forth in §§ 1 16(e) or §§ 1. 1 6(1). 

(it) The payment of any small entity fee other than those set forth in paragraph (c)(3) of this 

section (whether in the exact fee amount or not) will not be treated as a written 
assertion of entitlement to small entity status and will not be sufficient to establish small 
entity status in an application or a patent " 

3. [ X ] A copy of the International application as filed (35 U.S.C. 371(c)(2)): 



NOTE- Section J. 495 (b) was amended to require that the basic national fee and a copy of the international application must 
be filed with the Office by 30 months from the priority date to avoid abandonment "The International Bureau normally 
provides the copy of the international application to the Office in accordance with PCT Article 20 At the same time, 
the International Bureau notifies applicant of the communication to the Office In accordance with PCT Rule 47 1 , 
that notice shall be accepted by all designated offices as conclusive evidence that the communication has duly taken 
place. Thus, if the applicant desires to enter the national stage, the applicant normally need only check to be sure the 
notice from the International Bureau has been received and then pay the basic national fee by 30 months from the 
priority date " Notice of Jan. 7, 1993, 1147 O.G. 29 to 40, at 35-36 See item 14c below 



a. 


[ ] 


is transmitted herewith. 


b. 


[ ] 


is not required, as the application was filed with the United States Receiving 






Office. 


c. 


[X 3 


has been transmitted 




i. 


[ X ] by the International Bureau'. 






rw nf mailing of the. application ifrom form PCT/IB/308): 



[ 3 by applicant on 



[ X ] A translation of the International application into the English language (35 U.S.C. 
371(c)(2)): 

a. [ ] is transmitted herewith. 

b. [X] is not required as the application was filed in English. 

c. [ 3 was previously transmitted by applicant on . . 

Date 

d. [ ] will follow. 
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5. [X ] Amendments to the claims of the International application under PCT Article 19 (35 

U.S.C. 371(c)(3)): 

NOTE- The Notice of January 7, 1 993 points out that 37 C.F R § 1 .495(a) was amended to clarify the existing and continuing 
practice that PCT Article 19 amendments must be submitted by 30 months from the priority date and this deadline 
may not be extended The Notice further advises that "The failure to do so will not result in loss of the subject matter 
of the PCT Article 19 amendments. Applicant may submit that subject matter in a preliminary amendment filed under 
section 1.121. In many cases, filing an amendment under section 1 . 121 is preferable since grammatical or idiomatic 
errors may be corrected " 1147 O.G. 29-40, at 36 



[ ] are transmitted herewith. 
[ ] have been transmitted 

i. [ ] by the International Bureau. 

Date of mailing of the amendment (from form PCT/IB/308): 

ii. [ ] by applicant on . 

Date 

[X ] have not been transmitted as 

i. [X ] applicant chose not to make amendments under PCT Article 1 9. 

Date of mailing of Search Report (from form PCT/TSA/21 0): . 



[ ] the time limit for the submission of amendments has not yet expired. 
The amendments or a statement that amendments have not been 
made will be transmitted before the expiration of the time limit under 
PCT Rule 46.1. 



[X ] A translation of the amendments to the claims under PCT Article 19 (38 U.S.C. 
371(c)(3)): 

a. [ ] is transmitted herewith. 

b. [ ] is not required as the amendments were made in the English language. 

c. [ X ] has not been transmitted for reasons indicated at point 5(c) above. 

[ X ] A copy of the international examination report (PCT/IPEA/409) 
[X ] is transmitted herewith. 

[ ] is not required as the application was filed with the United States Receiving 
Office. 



[ ] Annex(es) to the international preliminary examination report 

a. [ ] is/are transmitted herewith. 

b. [ ] is/are not required as the application was filed with the United States 

Receiving Office. 

[ ] A translation of the annexes to the international preliminary examination report 

a. [ ] is transmitted herewith. 

b. [ ] is not required as the annexes are in the English language. 
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10. [X ] An oath or declaration of the inventor (35 U.S.C. 371(c)(4)) complying with 35 

U.S.C. 115 

a. [ ] was previously submitted by applicant on . 

Date 

b. [ X ] is submitted herewith, and such oath or declaration 

i. [ ] is attached to the application. 

ii. [X ] identifies the application and any amendments under PCT Article 1 

that were transmitted as stated in points 3(b) or 3(c) and 5(b); and 
states that they were reviewed by the inventor as required by 37 
C.F.R. 1.70. 

c. [ ] will follow. 

Other document(s) or information included: 

11. [ ] An International Search Report (PCT/ISA/210) or Declaration under PCT Article 

17(2)(a): 

a. [ ] is transmitted herewith. 

b. [ ] has been transmitted by the International Bureau. 

Date of mailing (from form PCT/IB/308): . 

c. [ ] is not required, as the application was searched by the United States 

International Searching Authority. 

d. [ ] will be transmitted promptly upon request. 

e. [ ] has been submitted by applicant on . 

Date 

12. [ ] An Information Disclosure Statement under 37 C.F.R. 1.97 and 1.98: 

a. [ ] is transmitted herewith. 

Also transmitted herewith is/are: 
[ ] Form PTO-1449(PTO/SB/08A and 08B). 
[ ] Copies of citations listed. 

b. [ ] will be transmitted within THREE MONTHS of the date of submission of 

requirements under 35 U.S.C. 371(c). 

c. [ ] was previously submitted by applicant on . 



13. [X ] An assignment document is transmitted herewith for recording. 

A separate [X ] "COVER SHEET FOR ASSIGNMENT (DOCUMENT) ACCOMPANYING 
NEW PATENT APPLICATION" or [ ] FORM PTO 1 595 is also attached. 

ORBOTECH LTD.; 

P.O. BOX 215. Yavne 81 102. Israel 
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[X ] Additional documents: 

a. [ ] Copy of request (PCT/RO/1 01) 

b. [ X ] International Publication No. WO 01/07893 

i. [ X ] Specification, claims and drawing 

ii. [ ] Front page only 

c. [] Preliminary amendment (37 C.F.R. § 1.121) 

d. [ ] Other 



[X ] The above checked items are being transmitted 

a. [ X ] before 30 months from any claimed priority date. 

b. [ ] after 30 months. 

[ ] Certain requirements under 35 U.S.C. 371 were previously submitted by the 
applicant on 

_, namely: 



AUTHORIZATION TO CHARGE ADDITIONAL FEES 

<NG: Accurately count claims, especially multiple dependent claims, to avoid unexpected high charges if extra 

claims are authorized 

-A written request may be submitted in an application that is an authorization to treat any concurrent or future reply, 
requiring a petition for an extension of time under this paragraph for its timely submission, as incorporating a petition 
for extension of time for the appropriate length of time An authorization to charge all required fees, fees under §1.17, 
or all required extension of time fees will be treated as a constructive petition for an extension of time in any 
concurrent or future reply requiring a petition for an extension of time under this paragraph for its timely submission 
Submission of the fee set forth in § 1 1 7(a) will also be treated as a constructive petition for an extension oj tunc in 
any concurrent reply requiring a petition for an extension of time under this paragraph for its timely submission " 
37C.FR § 1 136(a)(3) 

-Amounts of twenty-five dollars or less will not be returned unless specifically requested within a reasonable time, 
nor will the payer be notified of such amounts, amounts over twenty-five dollars may be returned by check or, if 
requested, by credit to a deposit account " 37 C F R. § 1.26(a) 



[X] The Commissioner is hereby authorized to charge the following additional fees that 
may be required by this paper and during the entire pendency of this application to 
Account No. 12-0425 

[X] 37 C.F.R. 1 .492(a)(1), (2), (3), and (4) (filing fees) 

WARNING: Because failure to pay the national fee within 30 months without extension (37 C F R § 1 495(b)(2)) 

results in abandonment of the application, it would be best to always check the above box. 

[ ] 37 C.F.R. 1.492(b), (c) and (d) (presentation of extra claims) 

NOTE Because additional fees for excess or multiple dependent claims not paid on filing or on later presentation must only 



(Transmittal Letter to the United States Elected Office (EO/US)— page 8 of 9) 13-18 



531 Rec'd PCT/PT' 14 JAN 2002 



be paid or these claims cancelled by amendment prior to the expiration of the time period set for response by the PTO 
in any notice of fee deficiency (37 C F R. § 1.492(d)), it might be best not to authorize the PTO to charge additional 
claim fees, except possible when dealing with amendments after final action 

[ ] 37 C.F.R. 1.17 (application processing fees) 
[ ] 37 C.F.R. 1.1 7(a)(l)-(5)(extension fees pursuant to § 1.136(a). 
[ ] 37 C.F.R. 1.1 8 (issue fee at or before mailing of Notice of Allowance, 
pursuant to 37 C.F.R. 1 .3 1 1(b)) 

NOTE- Where an authorization to charge the issue fee to a deposit account has been filed before the mailing of a Notice of 
Allowance, the issue fee will be automatically charged to the deposit account at the time of mailing the notice of 
allowance 37 C.F.R § 1.311(b) 

NOTE. 37C.FR. 1 28(b) requires "Notification of any change in loss of entitlement to small entity status must be filed in the 
application . . . prior to paying, or at the time of paying . . issue fee. " From the wording of 37 C.F.R § 1 28(b) ' (a) 
■ notification of change of status must be made even if the fee is paid as "other than a small entity" and (b) no 
notification is required if the change is to another small entity 

[ ] 37 C.F.R. § 1 .492(e) and (f) (surcharge fees for filing the declaration and/or 
filing an English translation of an InternatipnalAjffpJication later than 30 
months after the priority date). "~\*TT ( 




Reg. No.: 20,302 Julian H. Cohen 

(type or print name of practitioner) 

Tel. No.: (212) 708-1887 



Customer No.: 00140 



c/o Ladas & Parry 
26 West 61 st Street 
New York, N.Y. 10023 
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Commissioner of Patents & Trademarks 
Box Assignments 
Washington, D.C. 20231 



CERTIFICATE OF MAILING/TRANSMISSION (37 C 

I hereby certify that this correspondence is, on the date shown below J3e7^g: 
37C.F.R. 1.10* / / . 



as "Express Mail Post Office to Addressee" 
Mailing Label No. EV01 1020142US 
(mandatory) 



Date: January 14. 2002 



Signature 
CONNIE YANNOTTI 



(type or print name of person certifying) 



(Assignment (Document) Cover Sheet — page 1 of 2) 16-6a 
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| CHAPTER n 
IN THE UNITED STATES ELECTED OFFICE (EO/US) 

PCT/ILOO/00434 23 JULY 2000 25 JULY 1999 

INTERNATIONAL APPLICATION NO. INTERNATIONAL FILING DATE PRIORITY DATE CLAIMED 

OPTICAL INSPECTION SYSTEM 

TITLE OF INVENTION 

1. Uri GOLD; 2. Eli PARENTE; 3. Tally GILAT-BERNSHTEIN; 4. Edward BARANOVSKY 

5. TamirMARGALIT; 

APPLICANT(S) 

Box PCT 

Assistant Commissioner for Patents 
Washington, D.C. 20231 
ATTENTION: EO/US 

PRELIMINARY AMENDMENT 

Please amend the above identified application as follows: 
IN THE CLAIMS : 

Please cancel claims 1-215. 

Please add new claims 216-439 as follows: 



CERTIFICATE UNDER 37 1.10 



I hereby certify that this paper is being deposited with the United States Postal Service on this 
date JANUARY 14. 2002 in an envelope as "EXPRESS MAIL POST OFFICE TO 
ADDRESSEE" Mailing Label Number EVP 1 10201 42US addressed toj^feTAssistant 
Commissioner for Patents, Washington, D.C. 20231 




(Signature of person mailing paper) 



NOTE: Each paper or fee referred to as enclosed herein has the number of the "EXPRESS 
MAIL" mailing label place thereon prior to mailing 37 CFR 1 .16(b). 
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216. A method for determining a location of a border in a color image, the image 
comprising at least two color populations, between a first color region associated with a 
first one of the two color populations and a second color region associated with a second 
one of the two color populations, both the first color region and the second color region 
being comprised in the color image, the method comprising: 

identifying an approximate border location between said first color region and said 
second color region; 

determining a plurality of candidate border locations between the first color region 
and the second color region, each of said plurality of candidate border locations being 
determined by applying a corresponding border location method chosen from among a 
plurality of border location methods; 

choosing one method from among the plurality of border location methods as a 
preferred method; and 

determining a location of a border between said first color region and said second 
color region by designating one of said plurality of candidate border locations associated 
with the preferred method as the border. 

217. A method according to claim 216 and wherein said color image comprises an 
image of a patterned article. 

218. A method according to claim 217 and wherein said color image comprises a 
color image of an electrical circuit. 

219. A method according to claim 218 and wherein said color image comprises a 
color image of a lead frame. 

220. A method according to claim 216 and wherein the preferred border location 
method is chosen based on a rule for choosing a preferred border location method for a 
predetermined combination of color populations. 
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221. A method according to claim 216 and wherein the preferred border location 
method is chosen based on comparison of the border location result from applying at least 
two border location methods. 

222. A method for determining a location of a border in a color image, the image 
comprising at least two color populations, between a first color region associated with a 
first one of the two color populations and a second color region associated with a second 
one of the two color populations, both the first color region and the second color region 
being comprised in the color image, the method comprising: 

providing a border analysis map comprising, for each distinct pair of color 
populations, an indication of a preferred method for identifying the location of a border 
between said color populations; 

identifying an approximate border location between said first color region and said 
second color region; and 

determining a location of a border between said first color region and said second 
color region by using the preferred method indicated, in the border analysis map, for 
identifying the location of a border between said first color population and said second 
color population. 

223. A method according to claim 222 and wherein said color image comprises an 
image of a patterned article. 

224. A method according to claim 223 and wherein said color image comprises a 
color image of an electrical circuit. 

225. A method according to claim 223 and wherein said color image comprises a color 
image of a lead frame. 

226. A method according to claim 224 and wherein said electrical circuit comprises a 
printed circuit board. 
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227. A method according to claim 222 and wherein the color image comprises a color 
image of an object, and 

each said color population is associated with one material of a plurality of 
available materials 

228. A method according to claim 222 and wherein said color image also comprises a 
third color region associated with a third color population, and 

said identifying step comprises, for each two color regions among the first color 
region, the second color region, and the third color region which border each other, 
identifying an approximate border location between said two color regions, and 

said determining step comprises, for each said two color regions, determining a 
location of a border between said two color regions by using the preferred method 
indicated, in the border analysis map, for identifying the location of a border between the 
two color populations associated with said two color regions. 

229. A method according to claim 228 and also comprising: 
defining a window having a window size; 

examining a portion of said color image falling within said window to determine 
the number of distinct color regions at least a part of which are included in said portion; 
and 

if the number of distinct color regions is greater than two, classifying all borders 
within the window as belonging to a junction and determining a location of each border 
within said window using a preferred method for determining the location of a border 
belonging to a junction. 

230. A method according to claim 229 and wherein the preferred method for 
determining the location of a border belonging to a junction comprises a high-water-mark 
method. 

231. A method according to claim 222 and wherein the color image is represented 
using a color image representation method having a plurality of color defining 
characteristics, and 
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each said preferred method is selected from a plurality of available methods, and 
the plurality of available methods includes, for each color defining characteristic 
within the plurality of color defining characteristics, a method comprising performing an 
edge locating method in a component of the color image associated with said color defining 
characteristic. 

232. A method according to claim 231 and wherein said performing an edge locating 
method in a component comprises performing a sub-pixel contour-element locating 
method. 

233. A method according to claim 231 and wherein said performing an edge locating 
method only in a component comprises performing a high-water-mark method. 

234. A method according to claim 231 and wherein said color image representation 
method comprises an RGB method and said plurality of color defining characteristics 
comprises an R component, a G component, and a B component. 

235. A method according to claim 231 and wherein said color image representation 
method comprises an HSI method and said plurality of color defining characteristics 
comprises an H component, and S component, and an I component. 

236. A method for producing a color morphology map from a color image, the method 
comprising: 

providing a color image; 

reducing the color image to produce a reduced image comprising a plurality of 
color populations; 

determining borders between adjacent ones of the plurality of color populations in 
the reduced image; 

segmenting the reduced image to produce a binary image comprising borders and 
at least one non-border region; 

producing a skeleton image of the at least one non-border region, said skeleton 
image comprising a plurality of portions; and 
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assigning to each portion of the skeleton a color identity corresponding to one of 
the color populations. 

237. A method according to claim 236 and wherein the assigning step includes 
assigning a color population as a function of the spatial location of a portion of the skeleton 
relative to the color population. 

238. A method according to claim 237 and wherein the assigning step further includes 
superimposing the skeleton image on the reduced image. 

239. A method according to claim 236 and wherein the color image comprises a 24 bit 
color image. 

240. A method according to claim 236 and wherein the reduced image comprises a 3 bit 
image, and 

the plurality of color populations comprises at most 8 color populations. 

241. A method according to claim 236 and wherein said determining step comprises 
determining borders to an accuracy of one pixel. 

242. A method according to claim 236 and also comprising: 

identifying the presence of at least one morphological feature based, at least in 
part, on a result of the method. 

243. A method according to claim 236 and wherein the color image comprises an 
image of an patterned article. 

244. A method according to claim 243 and wherein the patterned article comprises an 
electrical circuit. 

245. A method for determining color contour elements (eels) in a color image, the 
method comprising: 
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providing a color image; 

reducing the color image to produce a reduced image comprising a plurality of 
color populations; 

determining borders between adjacent ones of the plurality of color populations, 
said borders comprising a plurality of border segments each border segment being 
surrounded by two color populations; and 

for each border segment, assigning a value to the border segment based, at least in 
part, on the two color populations surrounding the border segment. 

246. A method according to claim 245 and wherein the color image comprises a 24 bit 
color image. 

247. A method according to claim 245 and wherein the reduced image comprises a 3 bit 
image. 

248. A method according to claim 245 and wherein said determining step comprises 
determining borders to an accuracy of one pixel. 

249. A method according to claim 245 and wherein said determining step comprises 
determining borders to an accuracy of less than one pixel. 

250. A method according to claim 245 and wherein each said border segment has a first 
side and a second side, and 

said assigning step includes assigning a value based, at least in part, on the color 
population on the first side of said border segment and on the color population on the 
second side of said border segment. 

251. A method according to claim 245 and wherein the color image comprises an 
image of an patterned article. 

252. A method according to claim 251 and wherein the patterned article comprises an 
electrical circuit. 
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253. A method for identifying irregularities in a smooth curve in an image, the method 
comprising: 

identifying an ordered plurality of points along the smooth curve, each of said 
ordered plurality of points having at least one neighboring point in the ordered plurality of 
points; 

determining a normal direction to the smooth curve at each of said ordered 
plurality of points; and 

identifying a point of said ordered plurality of points as being part of an 
irregularity based, at least in part, on a local normal difference comprising a difference 
between the normal direction to the smooth curve at said point and a normal direction to 
the smooth curve at said at least one neighboring point of said point. 

254. A method according to claim 253 and wherein the image comprises an image of a 
patterned article. 

255. A method according to claim 254 and wherein said image comprises an image of 
an electrical circuit. 

256. A method according to claim 253 and wherein said local normal difference 
comprises a difference in direction of more than a predetermined angle. 

257. A method according to claim 256 and wherein said predetermined angle comprises 
an angle of approximately 22 degrees. 

258. A method according to claim 253 and wherein said image comprises a multiplicity 
of pixels and each of said ordered plurality of points is associated with one pixel. 

259. A method according to claim 258 and wherein each pixel comprised in said 
smooth curve is associated with one of said ordered plurality of points. 

260. A method according to claim 258 and wherein only some pixels comprised in said 
smooth curve are associated with one of said ordered plurality of points. 
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261 . A method according to claim 260 and wherein said some pixels are chosen so that 
a fixed number of pixels not associated with one of said ordered plurality of points falls 
between each one of said ordered plurality of points and each said at least one neighbor of 
each one of said ordered plurality of points. 

262. A method according to claim 253 and also comprising: 

determining a feature depth and feature aperture for said irregularity; and 
classifying said feature based, at least in part, on a result of said determining step. 

263. A method according to claim 262 and where said classifying includes classifying 
into one of a predetermined set of categories comprising at least one of the following 
categories: 

a nick; 

a protrusion; and 
a contour. 

264. A method according to claim 253 for detecting an irregularity in an electrical 
circuit, wherein said smooth curve represents a border between two materials in an image 
of the electrical circuit. 

265. A method for classifying an irregularity in an image of an object, the method 
comprising: 

determining a feature depth and feature aperture for said irregularity; and 
classifying said feature based, at least in part, on a result of said determining step. 

266. A method according to claim 265 and wherein said object comprises an electrical 
circuit. 

267. A method according to claim 265 and where said classifying includes classifying 
into one of a predetermined set of categories comprising at least one of the following 
categories: 

a nick; 



JL D Oi 3 Q S -.I-S ... -G .'.I. 3 -l-0;k:£' 



a protrusion; and 
a contour. 

268. A method for identifying a defect in a portion of an image of an object, the portion 
being close to a location in the image identified as an edge, the method comprising: 

identifying an excluded direction associated with the edge; and 
searching for defects, close to the location identified as an edge, in a plurality of 
directions not including the excluded direction. 

269. A method according to claim 268 and wherein said object comprises an electrical 
circuit. 

270. A method according to claim 268 and wherein the excluded direction comprises a 
direction perpendicular to the edge. 

271 . A method according to claim 268 and wherein said defect comprises a scratch. 

272. A method according to claim 268 and wherein said defect comprises a stain. 

273. A method for identifying a surface defect in an image of an object, the image 
comprising a multiplicity of pixels, the method comprising: 

choosing a region in which a surface defect is to be identified; 

subsampling the region thereby producing a subsampled image of the region; 

analyzing the subsampled image and identifying therefrom a surface defect in the 

region. 

274. A method according to claim 273 and wherein said image comprises a color image 
of an electrical circuit. 

275. A method according to claim 273 and wherein said subsampling comprises 
subsampling non-adjacent pixels. 
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276. A method according to claim 275 and wherein separation of non-adjacent pixels 
respective of adjacent pixels comprises a subsampling ratio. 

277. A method according to claim 276 and wherein the subsampling ratio is between 
approximately 1:13 and approximately 1:49. 

278. A method according to claim 273 and wherein said subsampling comprises 
subsampling in accordance with an approximate average distance between subsampled 
pixels. 

279. A method according to claim 278 and wherein said approximate average distance 
between subsampled pixels is between approximately 5 pixels and approximately 1 1 pixels. 

280. A method according to claim 273 and wherein the subsampling step comprises 
subsampling a portion of the region, and 

the subsampling step is performed a plurality of times using a plurality of portions 
of the region. 

281. A method according to claim 280 and wherein at least two of the plurality of 
portions of the region comprise partially overlapping portions of the region. 

282. A method for applying an image processing operator to a pixel in an array of 
pixels to produce a value, the method comprising: 

providing an n x n spread of a central pixel; 

if the central pixel comprises an edge pixel, assigning the value 0 to a result; 
if the central pixel does not comprise an edge pixel, performing the following 
steps for each one pixel in the n x n spread: 

if the one pixel is not an edge pixel, choosing the one pixel as a chosen 

pixel; 

if the one pixel is an edge pixel and a mirror image of the one pixel within 
the n x n spread is not an edge pixel, choosing the mirror image as the chosen pixel, 
otherwise choosing the central pixel as the chosen pixel; and 
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adding to the sum a product of the value of the chosen pixel and the result of 
applying a Gaussian operator to the central pixel; and 

dividing the sum by the square of (n+1) to produce a result. 

283. A method according to claim 282 and wherein n is equal to 3. 

284. A method for computing a gradient with a convoluted operator comprising: 
providing an image processing operator in the form of an n x n array of values, 

wherein n is an odd integer and the central value of the n x n array may take on either a 
positive value or a negative value; 

applying the image processing operator with the central value having a positive 
value to produce a first intermediate result; 

applying the image processing operator with the central value having a negative 
value to produce a second intermediate result; and 

summing the first intermediate result and the second intermediate result to produce 
a final result. 

285. A method according to claim 284 and also comprising: 

comparing the arithmetic sign of the first intermediate result and the arithmetic 
sign of the second intermediate result; and 

determining the presence of a local extremum based on a result of the comparing 

step. 

286. A method according to claim 285 and wherein said determining comprises 
determining with less than complete certainty. 

287. A method according to claim 284 and wherein n is equal to 5. 

288. An automated optical inspection device suitable for inspection of patterned articles 
comprising: 

at least one detector providing a polychromatic image output of at least a portion 
of a patterned article having a plurality of elements; and 
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processing circuitry receiving said polychromatic image output and providing at 
least one of the following inspection functionalities: 

inspection of a metal coating on at least one of the elements in the plurality 
of elements; and 

inspection of an at least partially transparent coating on at least one element 
in the plurality of elements. 

289. An automated optical inspection device suitable for inspection of patterned articles 
comprising: 

at least one detector providing a polychromatic image output of at least a portion 
of a patterned article comprising a plurality of elements; and 

processing circuitry receiving said polychromatic image output and providing the 
following inspection functionalities: 

inspection of a metal coating on at least one element in the plurality of 

elements; and 

inspection of an at least partially transparent coating on at least one element 
in the plurality of elements. 

290. An automated optical inspection device according to claim 288 and wherein said 
processing circuitry receiving said polychromatic image output provides the following 
additional inspection functionality: 

detection of residues on a surface of said patterned article. 

291. An automated optical inspection device according to claim 289 and wherein said 
processing circuitry receiving said polychromatic image output provides the following 
additional inspection functionality: 

detection of residues on a surface of said patterned article. 

292. An automated optical inspection device according to claim 288 and wherein said 
functionality of inspection of a metal coating on at least one of the elements in the plurality 
of elements comprises at least one of the following inspection modalities: 

detection of voids in the metal coating; 
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detection of surface irregularities in the metal coating; 
detection of scratches in the metal coating; 

detection of the presence of foreign materials on the metal coating; 
detection of stains on the metal coating; 
detection of oxidation of the metal coating; and 
detection of the shape of the metal coating. 

293. An automated optical inspection device according to claim 289 and wherein said 
functionality of inspection of a metal coating on at least one of the elements in the plurality 
of elements comprises at least one of the following inspection modalities: 

detection of voids in the metal coating; 

detection of surface irregularities in the metal coating; 

detection of scratches in the metal coating; 

detection of the presence of foreign materials on the metal coating; 
detection of stains on the metal coating; 
detection of oxidation of the metal coating; and 
detection of the shape of the metal coating. 

294. An automated optical inspection device according to claim 288 and wherein said 
functionality of inspection of an at least partially transparent coating on at least one of the 
elements in the plurality of elements comprises at least one of the following inspection 
modalities: 

detection of voids in the at least partially transparent coating; 

detection of surface irregularities in the at least partially transparent coating; 

detection of scratches in the at least partially transparent coating; 

detection of the presence of foreign materials on the at least partially transparent 

coating; 

detection of stains on the at least partially transparent coating; and 
detection of the shape of the at least partially transparent coating. 

295. An automated optical inspection device according to claim 289 and wherein said 
functionality of inspection of an at least partially transparent coating on at least one of the 
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elements in the plurality of elements comprises at least one of the following inspection 
modalities: 

detection of voids in the at least partially transparent coating; 

detection of surface irregularities in the at least partially transparent coating; 

detection of scratches in the at least partially transparent coating; 

detection of the presence of foreign materials on the at least partially transparent 

coating; 

detection of stains on the at least partially transparent coating; and 
detection of the shape of the at least partially transparent coating. 

296. An automated optical inspection device according to claim 288 and wherein said 
functionality of detection of residues on a surface of said patterned article comprises at 
least one of the following detection modalities: 

detection of surface irregularities on a surface of said patterned article; 
detection of the presence of foreign materials on a surface of said patterned article; 
detection of stains on a surface of said patterned article; and 
detection of the shape of marks on a surface of said patterned article. 

297. An automated optical inspection device according to claim 289 and wherein said 
functionality of detection of residues on a surface of said patterned article comprises at 
least one of the following detection modalities: 

detection of surface irregularities on a surface of said patterned article; 
detection of the presence of foreign materials on a surface of said patterned article; 
detection of stains on a surface of said patterned article; and 
detection of the shape of marks on a surface of said patterned article. 

298. An automated optical inspection device according to claim 288 and wherein the 
patterned article comprises an electrical circuit. 

299. An automated optical inspection device according to claim 289 and wherein the 
patterned article comprises an electrical circuit. 
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300. An automated inspection device according to claim 298 and wherein the elements 
comprise conductors in the electrical circuit. 

301. An automated inspection device according to claim 299 and wherein the elements 
comprise conductors in the electrical circuit. 

302. An automated optical inspection device according to claim 300 and wherein the 
electrical circuit comprises a printed circuit board. 

303. An automated optical inspection device according to claim 301 and wherein the 
electrical circuit comprises a printed circuit board. 

304. An automated optical inspection device according to claim 300 and wherein the 
electrical circuit comprises a ball grid array substrate. 

305. An automated optical inspection device according to claim 301 and wherein the 
electrical circuit comprises a ball grid array substrate. 

306. An automated optical inspection device according to claim 288 and wherein the 
patterned article comprises a lead frame. 

307. An automated optical inspection device according to claim 289 and wherein the 
patterned article comprises a lead frame. 

308. An automated optical inspection device according to claim 288 and wherein the 
metal coating comprises a metal plating. 

309. An automated optical inspection device according to claim 289 and wherein the 
metal coating comprises a metal plating. 

310. An automated optical inspection device according to claim 304 and wherein the 
elements comprise balls in the ball grid array substrate. 
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311. An automated optical inspection device according to claim 305 and wherein the 
elements comprise balls in the ball grid array substrate. 

312. An automated optical inspection device according to claim 288 wherein the 
patterned article comprises an etched metal substrate. 

313. An automated optical inspection device according to claim 289 wherein the 
patterned article comprises an etched metal substrate. 

314. An automated optical inspection device according to claim 288 and wherein the 
patterned article comprises an engraved metal substrate. 

315. An automated optical inspection device according to claim 289 and wherein the 
patterned article comprises an engraved metal substrate. 

316. An automated optical inspection device suitable for inspection of an article and 
comprising: 

at least one detector providing a polychromatic image output of at least a portion 
of an article; and 

processing circuitry receiving said polychromatic image output and providing 
edge detection to sub-pixel accuracy operative to distinguish borders among regions of 
different color. 

317. An automated optical inspection device according to claim 316 and wherein said 
processing circuitry is operative to distinguish a border between a first color region 
associated with a first color population and a second color region associated with a second 
color population by identifying an approximate border location between said first color 
region and said second color region and determining a location of a border between said 
first color region and said second color region by using a preferred method for identifying 
the location of a border between said first color population and said second color 
population. 
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318. An automated optical inspection device according to claim 317 and wherein said 
polychromatic image output comprises a color image represented using a color image 
representation method having a plurality of color defining characteristics, and 

each said preferred method is selected from a plurality of available methods, and 
the plurality of available methods includes, for each color defining characteristic 
within the plurality of color defining characteristics, a method comprising performing an 
edge locating method in a component of the color image associated with said color defining 
characteristic. 

319. An automated optical inspection device according to claim 318 and wherein the 
edge locating method comprises a sub-pixel contour-element locating method. 

320. An automated optical inspection device suitable for inspection of an article and 
comprising: 

at least one detector providing an image output of at least a portion of an article 
having at least three different regions, each of which regions is distinguished at least by an 
optical characteristic detectable by said detector; and 

processing circuitry receiving said image output and providing edge detection to 
sub-pixel accuracy operative to distinguish borders of at least two of said at least three 
different regions. 

321. An automated optical inspection device according to claim 320 and wherein said 
processing circuitry receives said image output and provides edge detection to sub-pixel 
accuracy operative to distinguish borders of said at least three different regions. 

322. An automated optical inspection device according to claim 320 and wherein said 
processing circuitry provides mapping based at least partially on said image output which 
identifies materials in each of said different regions having a common border. 

323. An automated optical inspection device suitable for inspection of an article and 
comprising: 

at least one detector providing an image output of at least a portion of an article; 

and 
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processing circuitry receiving said image output and providing mapping based at 
least partially on said image output which identifies materials in each of different regions 
of said at least a portion of said article having a common border. 

324. An automated optical inspection device according to claim 323 and wherein said 
processing circuitry is operative to sub-pixel accuracy. 

325. An automated optical inspection device suitable for inspection of an article 
comprising: 

at least one detector providing a polychromatic image output of at least a portion 
of an article; 

processing circuitry receiving said polychromatic image output and providing at 
least one of the following inspection functionalities: 

segmentation of said polychromatic image into a segmented color image 
comprising a plurality of color regions each having one of a plurality of representative 
color possibilities; 

binarization of the segmented color image into a binary image including regions 
and borders; 

morphological erosion of the regions to provide a skeleton representing the 
regions; and 

assignment of color regions to elements in the skeleton. 

326. An automated optical inspection device according to claim 325 and wherein the 
processing circuitry is operative to superimpose the skeleton and the segmented color 
image, and to assign data representative of regions to elements in the skeleton as a function 
of the superimposition. 

327. An automated optical inspection device according to claim 326 and wherein the 
processing circuitry is operative to assign data representative of regions to elements in the 
skeleton the superimposed location of an element of the skeleton in a color region. 

328. An automated optical inspection device suitable for inspection of an article 
comprising: 
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at least one detector providing a polychromatic image output of at least a portion 
of an article; and 

processing circuitry receiving said polychromatic image output and operative to 
segment said polychromatic image into plurality of color regions chosen from among a 
predefined number of color region possibilities, and to generate a representation of the 
polychromatic image comprising color region data representing each color region in the 
image, and border data for representing each border among color regions. 

329. A color image representing a patterned article, the color image comprising a 
morphologically eroded polychromatic image of the patterned article. 

330. A color image according to claim 329 and wherein colors of portions of said 
morphologically eroded polychromatic image indicate materials at various regions of the 
patterned article. 

331. A color image according to claim 329 and wherein the patterned article comprises 
an electrical circuit. 

332. A color image according to claim 331 and wherein the elements comprise 
conductors in the electrical circuit. 

333. A color image according to claim 332 and wherein the electrical circuit comprises 
a printed circuit board. 

334. A color image according to claim 332 and wherein the electrical circuit comprises 
a ball grid array substrate. 

335. A color image according to claim 329 and wherein the patterned article comprises 
a lead frame. 

336. A color image according to claim 329 and wherein the metal coating comprises a 
metal plating. 
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337. A color image according to claim 334 and wherein the elements comprise balls in 
the ball grid array substrate. 

338. A color image according to claim 329 wherein the patterned article comprises an 
etched metal substrate. 

339. A color image according to claim 329 and wherein the patterned article comprises 
an engraved metal substrate. 

340. An automated optical inspection device suitable for inspection of an article 
comprising: 

at least one detector providing a polychromatic image output of at least a portion 
of an article; and 

processing circuitry receiving said polychromatic image output and providing a 
color image representing said article, the color image comprising a morphologically eroded 
polychromatic image of at least part of said article. 

341 . An automated optical inspection device according to claim 340 and wherein colors 
of portions of said morphologically eroded polychromatic image indicate materials at 
various regions of the article. 

342. A color image representing a patterned article, the color image comprising a map 
which indicates borders between different regions of an article and identifies materials in 
each of said different regions having a common border. 

343. A color image according to claim 342 and wherein colors of portions of said map 
indicate materials at said different regions. 

344. A color image according to claim 343 and wherein each color comprises one color 
from a predetermined possibility of colors. 



345. A color image according to claim 342 and wherein the patterned article comprises 
an electrical circuit. 
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346. A color image according to claim 345 and wherein the elements comprise 
conductors in the electrical circuit. 

347. A color image according to claim 346 and wherein the electrical circuit comprises 
a printed circuit board. 

348. A color image according to claim 346 and wherein the electrical circuit comprises 
a ball grid array substrate. 

349. A color image according to claim 342 and wherein the patterned article comprises 
a lead frame. 

350. A color image according to claim 342 and wherein the metal coating comprises a 
metal plating. 

351. A color image according to claim 348 and wherein the elements comprise balls in 
the ball grid array substrate. 

352. A color image according to claim 342 and wherein the patterned article comprises 
an etched metal substrate. 

353: A color image according to claim 342 and wherein the patterned article comprises 
an engraved metal substrate. 

354. An automated optical inspection device suitable for inspection of an article 
comprising: 

at least one detector providing a polychromatic image output of at least a portion 
of an article; and 

processing circuitry receiving said polychromatic image output and providing a 
color image representing said article, the color image comprising a map which indicates 
borders between different regions of an article and identifies materials in each of said 
different regions having a common border. 
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355. An automated optical inspection device according to claim 354 and wherein colors 
of portions of said map indicate materials at various regions of the article. 

356. An automated optical inspection device according to claim 354 and wherein said 
processing circuitry receives said image output and provides border indication to sub-pixel 
accuracy. 

357. An automated optical inspection device according to claim 355 and wherein each 
color comprises a color chosen from among a predetermined possibility of colors. 

358. An automated optical inspection device according to claim 354 and wherein the 
patterned article comprises an electrical circuit. 

359. An automated optical inspection device according to claim 358 and wherein the 
electrical circuit comprises a printed circuit board. 

360. An automated optical inspection device according to claim 359 and wherein the 
electrical circuit comprises a ball grid array substrate. 

361. An automated optical inspection device according to claim 354 and wherein the 
patterned article comprises a lead frame. 

362. An automated optical inspection device according to claim 354 and wherein the 
different materials include different metals. 

363. An automated optical inspection device according to claim 359 and wherein the 
different regions include uncoated metal conductors, coated metal conductors, uncoated 
substrate, and coated substrate. 

364. An automated optical inspection device according to claim 354 and wherein the 
patterned article comprises an etched metal substrate. 
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365. An automated optical inspection device according to claim 354 and wherein the 
patterned article comprises an engraved metal substrate. 

366. An automated optical inspection device suitable for inspection of a patterned 
article comprising: 

at least one detector providing an image output of at least a portion of a patterned 
article comprising at least three materials having detectable edges; and 

processing circuitry receiving said image output and providing an inspection 
functionality comprising binarization of the image output to provide a binary image 
including region image data and border image data. 

367. An automated optical inspection device according to claim 366 and wherein said 
patterned article comprises an electrical circuit. 

368. An automated optical inspection device according to claim 367 and wherein said 
electrical circuit comprises a printed circuit board. 

369. An automated optical inspection device according to claim 368 and wherein said 
printed circuit board comprises a ball grid array substrate. 

370. An automated optical inspection device according to claim 367 and wherein said 
patterned article comprises a lead frame. 

371. An automated optical inspection device according to claim 367 and wherein said 
patterned article comprises an etched metal substrate. 

372. An automated optical inspection device according to claim 367 and wherein the 
patterned article comprises an engraved metal substrate. 

373. An automated optical inspection device suitable for inspection of a patterned 
article comprising: 

at least one detector providing an image output of at least a portion of a patterned 
article comprising a plurality of materials having detectable borders between the plurality 
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of materials; and 

processing circuitry receiving said image output and providing an inspection 
functionality comprising pattern analysis of image data relating to the borders to provide an 
indication of the presence of nicks and protrusions along said borders. 

374. An automated optical inspection device according to claim 373 and wherein said 
patterned article comprises an electrical circuit. 

375. An automated optical inspection device according to claim 374 and wherein said 
patterned article comprises a printed circuit board. 

376. An automated optical inspection device according to claim 375 and wherein said 
printed circuit board comprises a ball grid array substrate. 

377. An automated optical inspection device according to claim 374 and wherein said 
patterned article comprises a lead frame. 

378. An automated optical inspection device according to claim 373 and wherein said 
plurality of materials comprises at least three materials. 

379. An automated optical inspection device according to claim 378 and wherein said 
materials include materials from the following group: uncoated metal conductors, coated 
material conductors, uncoated substrate, coated substrate. 

380. An automated optical inspection device according to claim 374 and wherein said 
patterned article comprises an etched metal substrate. 

381. An automated optical inspection device according to claim 374 and wherein the 
patterned article comprises an engraved metal substrate. 

382. An automated optical inspection device according to claim 373 and wherein said 
inspection functionality comprising pattern analysis includes: 
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identifying an ordered plurality of points along a border defined by the image data 
relating to the borders, each of said ordered plurality of points having at least one 
neighboring point in the ordered plurality of points; 

determining a normal direction to the smooth curve at each of said ordered 
plurality of points; and 

identifying a point of said ordered plurality of points as being part of an 
irregularity comprising one of a nick and a protrusion based, at least in part, on a local 
normal difference comprising a difference between the normal direction to the smooth 
curve at said point and a normal direction to the smooth curve at said at least one 
neighboring point of said point. 

383. An automated optical inspection device suitable for inspection of an article 
comprising: 

at least one detector providing an image output of at least a portion of an article 
comprising multiple materials having detectable borders between the materials; and 

processing circuitry receiving said image output and providing at least one of the 
following inspection functionalities: 

segmentation of said image output into a segmented image including region 
image data and border image data; 

analysis of the region image data using a first technique; and 
analysis of the border image data using a second technique, different from 
the first technique. 

384. An automated optical inspection device according to claim 383 and wherein said 
article comprises an electrical circuit. 

385. An automated optical inspection device according to claim 383 and wherein said 
first technique includes neighborhood analysis and wherein said second technique includes 
border following analysis for differentiating border-near pixels from non-border-near 
pixels. 



386. An automated optical inspection device suitable for inspection of an article 
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comprising: 

at least one detector providing a polychromatic image output of at least a portion 
of an article comprising multiple materials having detectable borders between the 
materials; and 

processing circuitry receiving said polychromatic image output and providing at 
least one of the following inspection functionalities: 

segmentation of said polychromatic image output into a segmented color 
image including color regions chosen from among a set of predetermined colors; 

analysis of the region image data relating to a first color using a first 

technique; and 

analysis of the region image data relating to a second color using a second 
technique, different from the first technique. 

387. An automated optical inspection device according to claim 386 and wherein said 
article comprises an electrical circuit. 

388. An automated optical inspection device according to claim 387 and wherein said 
electrical circuit comprises a printed circuit board. 

389. An automated optical inspection device according to claim 388 and wherein said 
printed circuit board comprises a ball grid array substrate. 

390. An automated optical inspection device according to claim 386 and wherein said 
article comprises a lead frame. 

391. An automated optical inspection device suitable for inspection of an article 
comprising: 

at least one detector providing an image output of at least a portion of an article 
comprising multiple materials having detectable borders between the materials; and 

processing circuitry receiving said image output and providing at least one of the 
following inspection functionalities: 

segmentation of said image output into a segmented image including region image 
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data and border image data; and 

smoothing the region image data in a manner not influenced by at least some of 
the border image data. 

392. An automated optical inspection device according to claim 391 and wherein said 
article comprises an electrical circuit. 

393. An automated optical inspection device according to claim 392 and wherein said 
electrical circuit comprises a printed circuit board. 

394. An automated optical inspection device according to claim 393 and wherein said 
printed circuit board comprises a ball grid array substrate. 

395. An automated optical inspection device according to claim 391 and wherein said 
article comprises a lead frame. 

396. An automated optical inspection device suitable for inspection of an electrical 
circuit comprising: 

at least one detector providing an image output of at least a portion of an electrical 
circuit comprising multiple materials; and 

processing circuitry receiving said image output and providing an output 
indication of the presence of at least one predetermined material. 

397. An automated optical inspection device according to claim 396 and wherein said 
electrical circuit comprises a printed circuit board. 

398. An automated optical inspection device according to claim 397 and wherein said 
printed circuit board comprises a ball grid array substrate. 

399. An automated optical inspection device according to claim 398 and wherein said 
electrical circuit comprises a flat panel display substrate. 
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400. An automated optical inspection device according to claim 396 and wherein said 
processing circuitry provides an output indication of the presence of at least two materials 
out of a group of at least three predetermined materials. 

401. An automated optical inspection device according to claim 396 and wherein said 
processing circuitry provides an output indication of the presence of at least three 
predetermined materials. 

402. An automated optical inspection device according to claim 396 and wherein said 
processing circuitry provides an output indication of the presence of copper. 

403. An automated optical inspection device according to claim 396 and wherein said 
processing circuitry provides an output indication of the presence of gold. 

404. An automated optical inspection device according to claim 396 and wherein said 
processing circuitry provides an output indication of the presence of a solder mask 
material. 

405. An automated optical inspection device according to claim 396 and wherein said 
processing circuitry provides an output indication of the presence of silver. 

406. An automated optical inspection device according to claim 396 and wherein said 
processing circuitry provides an output indication of the presence of photoresist residue. 

407. An automated optical inspection device according to claim 396 and wherein said 
processing circuitry provides an output indication of the presence of oxidized metal. 

408. An automated optical inspection device according to claim 396 and wherein said 
image output represents a portion of the electrical circuit as viewed through a translucent 
overlay. 

409. An automated optical inspection device suitable for inspection of a patterned 
article comprising: 
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at least one detector providing an image output of at least a portion of a patterned 
article through a translucent overlay at least partially covering the patterned article; and 

processing circuitry receiving said image output and providing an output 
indication which is independent of the translucent overlay. 

410. An automated inspection device according to claim 409 and wherein the patterned 
article comprises an electrical circuit. 

411. An automated inspection device according to claim 410 and wherein the electrical 
circuit comprises a printed circuit board. 

412. An automated inspection device according to claim 411 and wherein the printed 
circuit board comprises a ball grid array substrate. 

413. An automated inspection device according to claim 409 and wherein the 
processing circuitry is operative to provide an indication of parts of the pattern covered by 
the translucent overlay and parts of the pattern not covered by the translucent overlay. 

414. Apparatus for determining a location of a border in a color image, the image 
comprising at least two color populations, between a first color region associated with a 
first one of the two color populations and a second color region associated with a second 
one of the two color populations, both the first color region and the second color region 
being comprised in the color image, the apparatus comprising: 

a border identification unit operative to identify an approximate border location 
between said first color region and said second color region; 

a candidate border determination unit operative to determine a plurality of 
candidate border locations between the first color region and the second color region, each 
of said plurality of candidate border locations being determined by applying a 
corresponding border location method chosen from among a plurality of border location 
methods; 

a method selector operative to choose one method from among the plurality of 
border location methods as a preferred method; and 
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a border determination unit operative to determine a location of a border between 
said first color region and said second color region by designating one of said plurality of 
candidate border locations associated with the preferred method as the border. 

415. Apparatus for determining a location of a border in a color image, the image 
comprising at least two color populations, between a first color region associated with a 
first one of the two color populations and a second color region associated with a second 
one of the two color populations, both the first color region and the second color region 
being comprised in the color image, the apparatus comprising: 

a border analysis map unit operative to provide a border analysis map comprising, 
for each distinct pair of color populations, an indication of a preferred method for 
identifying the location of a border between said color populations; 

a border identification unit operative to identify an approximate border location 
between said first color region and said second color region; and 

a border determination unit operative to determine a location of a border between 
said first color region and said second color region by using the preferred method indicated, 
in the border analysis map, for identifying the location of a border between said first color 
population and said second color population. 

416. Apparatus for producing a color morphology map from a color image, the 
apparatus comprising: 

a reducing unit operative to reduce the color image to produce a reduced image 
comprising a plurality of color populations; 

a border determination unit operative to determine borders between adjacent ones 
of the plurality of color populations in the reduced image; 

a segmentation unit operative to segment the reduced image to produce a binary 
image comprising borders and at least one non-border region; 

a skeleton production unit operative to produce a skeleton image of the at least one 
non-border region, said skeleton image comprising a plurality of portions; and 

an assigning unit operative to assign to each portion of the skeleton a color identity 
corresponding to one of the color populations. 
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417. Apparatus for identifying irregularities in a smooth curve in an image, the 
apparatus comprising: 

an identification unit operative to identify an ordered plurality of points along the 
smooth curve, each of said ordered plurality of points having at least one neighboring point 
in the ordered plurality of points; 

a normal direction unit operative to determine a normal direction to the smooth 
curve at each of said ordered plurality of points; and 

an irregularity unit operative to identify a point of said ordered plurality of points 
as being part of an irregularity based, at least in part, on a local normal difference 
comprising a difference between the normal direction to the smooth curve at said point and 
a normal direction to the smooth curve at said at least one neighboring point of said point. 

418. Apparatus for classifying an irregularity in an image of an object, the apparatus 
comprising: 

a dimension determination unit operative to determine a feature depth and feature 
aperture for said irregularity; and 

a classification unit operative to classify said feature based, at least in part, on an 
output of said dimension determination unit. 

419. Apparatus for identifying a defect in a portion of an image of an object, the portion 
being close to a location in the image identified as an edge, the apparatus comprising: 

an excluded direction unit operative to identify an excluded direction associated 
with the edge; and 

a defects unit operative to search for defects, close to the location identified as an 
edge, in a plurality of directions not including the excluded direction. 

420. Apparatus for identifying a surface defect in a region of an image of an object, the 
image comprising a multiplicity of pixels, the apparatus comprising: 

a subsampling unit operative to subsample the region in accordance with a 
subsampling ratio, thus producing a subsampled image of the region; 

a defect identification unit operative to identify a surface defect in the subsampled 
image; and 
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a correspondence unit operative to identify a surface defect in the region 
corresponding to the surface defect in the subsampled image. 

421 . A method for inspection of patterned articles comprising: 

providing a polychromatic image output of at least a portion of a patterned article 
having a plurality of elements; and 

receiving said polychromatic image output and providing at least one of the 
following inspection functionalities: 

inspection of a metal coating on at least one of the elements in the plurality of 
elements; and 

inspection of an at least partially transparent coating on at least one element in the 
plurality of elements. 

422. A method for inspection of patterned articles comprising: 

providing a polychromatic image output of at least a portion of a patterned article 
comprising a plurality of elements; and 

receiving said polychromatic image output and providing the following inspection 
functionalities: 

inspection of a metal coating on at least one element in the plurality of elements; 

and 

inspection of an at least partially transparent coating on at least one element in the 
plurality of elements. 

423. A method for inspection of an article comprising: 

providing a polychromatic image output of at least a portion of an article; and 
receiving said polychromatic image output and providing edge detection to sub- 
pixel accuracy operative to distinguish borders among regions of different color. 

424. A method for inspection of an article comprising: 

providing an image output of at least a portion of an article having at least three 
different regions, each of which regions is distinguished at least by an optical 
characteristic; and 
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receiving said image output and providing edge detection to sub-pixel accuracy 
operative to distinguish borders of at least two of said at least three different regions. 

425. A method for inspection of an article comprising: 

providing an image output of at least a portion of an article; and 

receiving said image output and providing mapping based at least partially on said 

image output which identifies materials in each of different regions of said at least a portion 

of said article having a common border. 

426. A method for inspection of an article comprising: 

providing a polychromatic image output of at least a portion of an article; and 
receiving said polychromatic image output and providing at least one of the 

following inspection functionalities: 

segmentation of said polychromatic image into a segmented color image 

comprising a plurality of color regions each having one of a plurality of representative 

color possibilities; 

binarization of the segmented color image into a binary image including regions 
and borders; 

morphological erosion of the regions to provide a skeleton representing the 
regions; and 

assignment of color regions to elements in the skeleton. 

427. A method for inspection of an article comprising: 

providing a polychromatic image output of at least a portion of an article; and 
receiving said polychromatic image output and providing a color image 

representing said article, the color image comprising a morphologically eroded 

polychromatic image of at least part of said article. 

428. A method for inspection of an article comprising: 

providing a polychromatic image output of at least a portion of an article; and 
receiving said polychromatic image output and providing a color image 
representing said article, the color image comprising a map which indicates borders 
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between different regions of an article and identifies materials in each of said different 
regions having a common border. 

429. A method for inspection of a patterned article comprising: 

providing an image output of at least a portion of a patterned article comprising at 
least three materials having detectable edges; and 

receiving said image output and providing an inspection functionality comprising 
binarization of the image output to provide a binary image including region image data and 
border image data. 

430. A method for inspection of a patterned article comprising: 

providing an image output of at least a portion of a patterned article comprising a 
plurality of materials having detectable borders between the plurality of materials; and 

receiving said image output and providing an inspection functionality comprising 
pattern analysis of image data relating to the borders to provide an indication of the 
presence of nicks and protrusions along said borders. 

43 1 . A method for inspection of an article comprising: 

providing an image output of at least a portion of an article comprising multiple 
materials having detectable borders between the materials; 

receiving said image output and providing at least one of the following inspection 
functionalities: 

segmentation of said image output into a segmented image including region image 
data and border image data; 

analysis of the region image data using a first technique; and 

analysis of the border image data using a second technique, different from the first 
technique. 

432. A method for inspection of an article comprising: 

providing a polychromatic image output of at least a portion of an article 
comprising multiple materials having detectable borders between the materials; and 

receiving said polychromatic, image output and providing at least one of the 
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following inspection functionalities: 

segmentation of said polychromatic image output into a segmented color image 
including color regions chosen from among a set of predetermined colors; 

analysis of the region image data relating to a first color using a first technique; 

and 

analysis of the region image data relating to a second color using a second 
technique, different from the first technique. 

433. A method for inspection of an article comprising: 

providing an image output of at least a portion of an article comprising multiple 
materials having detectable borders between the materials; and 

receiving said image output and providing at least one of the following inspection 
functionalities: 

segmentation of said image output into a segmented image including region image 
data and border image data; and 

smoothing the region image data in a manner not influenced by at least some of 
the border image data. 

434. A method for inspection of an electrical circuit comprising: 

providing an image output of at least a portion of an electrical circuit comprising 
multiple materials; and 

receiving said image output and providing an output indication of the presence of 
at least one predetermined material. 

435. A method for inspection of a patterned article comprising: 

providing an image output of at least a portion of a patterned article through a 
translucent overlay at least partially covering the patterned article; and 

receiving said image output and providing an output indication which is 
independent of the translucent overlay. 

436. A method for automatically optically inspecting color images of objects, 
comprising: 
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acquiring polychromatic images of an object; 

processing the polychromatic images and outputting a report indicating candidate 
defects and features chosen from among a predetermined set of features, and providing an 
image of a region surrounding each said candidate defect and feature. 

437. Apparatus for automatically optically inspecting color images of objects, 
comprising: 

a sensor operative to acquire polychromatic images of an object; 

an image processor operative to process the polychromatic images and output a 
report indicating candidate defects and features chosen from among a predetermined set of 
features, and to provide an image of a region surrounding each said candidate defect and 
feature. 

438. Apparatus according to claim 437 and wherein the processor includes a binary 
image processor operative to process binary images generated from the polychromatic 
image, and a color image processor operative to process the polychromatic images. 

439. Apparatus according to claim 437 and further comprising a morphological image 
processor operative to generate a skeleton map from a multicolor image derived from the 
polychromatic image. 
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OPTICAL INSPECTION SYSTEM 
FIELD OF THE INVENTION 
The present invention relates to apparatus and methods for analyzing 
5 images, especially but not exclusively color images, and more specifically to apparatus 
and methods of image analysis which are useful in inspection of a patterned object. 
BACKGROUND OF THE INVENTION 
Apparatus and methods for analyzing images, and in particular for image 
analysis which is useful in inspection of a patterned object, are well known in the art. 
10 The following references describe image processing methods which may 

be useful in understanding the present invention: 

C. Gonzalez and P. Wintz, Digital Imag e Processing. Addison Wesley, 
Reading, MA, 1987; and 

John C. Russ, The Image Pr ocessin g Handbook . CRC Press, 1 994. 
1 5 The following references describe edge detection methods which may be 

useful in understanding the present invention: 



Royal Society of London; and 

M. Chapron, "A new chromatic edge-detector used for color image 
20 segmentation", 1 1 th APR International Conference on Pattern Recognition. 

The following references describe color image segmentation methods 
which may be useful in understanding the present invention: 

Philippe Pujas and Marie-Jose Aldon, "Robust Colour Image 
Segmentation", 7 th International Conference on Advanced Robotics, San Filiu de 
25 Guixols, Spain, September 22, 1995, and 

Leila Shararenko, Maria Petrou, and Josef Kittler, "Automatic Watershed 
Segmentation of Randomly Textured Colour Images, IEEE. 

US Patent 4,758,888 to Lapidot describes a method of and means for 
inspecting workpieces traveling along a production line, including on-line inspection for 
30 flaws without interrupting the progress or workpieces along the production line. 

US Patent 5,058,982 to Katzir describes an illumination system 



D. Marr and E. Hildreth, 




l, Proceedings of the 
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particularly useful for inspection of a workpiece by an optical scanner. 

Israel Patent 81450, owned by Orbotech Ltd., Yavne, Israel, granted 
January 8, 1995 and entitled "Illumination Device for an Optical Scanner", describes an 
illumination device particularly suited for use with an optical scanner used in automatic 
inspection, especially for delivering intense, multi-directional light energy to a target 
area. 

US Patent 5,586,058 to Aloni et al. describes apparatus and methods for 
inspecting objects and detecting defects therein, including inspecting a binary level 
representation of the object, inspecting a gray level representation of the object, and 
preferably reinspecting the grayscale representation of the object to filter false alarms 
and to classify defects. 

US Patent 5,774,572 to Caspi describes an automatic visual inspection 
system, which is preferably operative to convolve a 2-dimensional digital gray scale 
image of an object with a filter function related to the second derivative of a Gaussian 
function forming a 2-dimensional convoluted image have signed values. The location of 
an edge in the object is achieved by finding zero crossings between adjacent oppositely 
signed values. 

US Patent 5,774,573 to Caspi et al. describes a visual inspection system 
which uses convolution of a 2-dimensional digital gray scale image of an object with a 
filter function related to the second derivative of a Gaussian function forming a 
2-dimensional convolved imaged having signed values. The convolution of Caspi et al. 
can be performed with a difference-of-two-Gaussians, one positive and one negative. 

PCT Application IL98/00393 describes inspection of printed circuit 
boards using color, including the use of color to identify certain types of conditions, 
such as conductor oxidation, using color. 

PCT Application IL98/00477 describes methods for analytically 
representing an image so that morphological operations such as dilation, erosion, and 
scale measurement may be performed on non-binary pixels, preferably more efficiently 
than using previous methods. 

Methods which may be useful in image analysis are described in the 
following publication: 
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Dorin Comaniciu and Peter Meer, "Distribution Free Decomposition of 
Multivariate Data", SPR '98 Invited Submission, Department of Electrical and 
Computer Engineering, Rutgers University, Piscataway, NJ 08855, USA. 

The disclosures of all references mentioned above and throughout the 
5 present specification are hereby incorporated herein by reference. 

SUMMARY OF THE INVENTION 
The present invention seeks to provide an improved apparatus and 
method for visually inspecting objects and analyzing their images, especially but not 

10 exclusively color images. The apparatus and methods of the present invention are 
believed to be particularly applicable to analyzing images of a patterned object, and 
more particularly, applicable to analyzing images of a patterned object which includes a 
plurality of elements that have characteristic optical attributes. Objects for which the 
apparatus and method of the present invention are particularly suited to inspect include 

15 ball grid array substrates ("BGA"), printed circuit board substrates, particularly printed 
circuit board substrates including multiple conductor materials, laminated printed circuit 
boards, lead frames, flat panel displays, hybrid chip packaging substrates, tape 
automated bonding substrates, and other similar multi-material patterned objects. 

All prior art object inspection systems suffer from certain limitations 

20 which make them useful for inspecting only certain types of objects, even if the objects 
being inspected are limited to the fields of printed circuit boards, integrated circuits, and 
similar electronic components. The present invention seeks to provide improved 
apparatus and methods of analyzing images, particularly color images, particularly for 
the purpose of improving the functionality of object inspection systems, increasing the 

25 diversity of objects that can be inspected by the system, and generally reducing the 
limitations thereof. 

It is believed that improved color processing may be particularly 
effective in analyzing images of multi-colored objects, particularly in cases where 
adjacent colored areas in the multi-colored objects have colors which are difficult to 
30 distinguish from each other in a monochrome image. For example, in the case of 
inspection of a ball grid array substrate or a laminated printed circuit board, it may be 
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desirable to distinguish between areas which are, on the surface thereof, gold and areas 
which are, on the surface thereof, copper, and to distinguish between metallic areas 
which are covered by partially transparent masks and areas which are uncovered. 
Similarly, areas which are, on the surface thereof, silver may be distinguished from 
5 other areas. It is believed that distinguishing between such areas may be accomplished 
more effectively using the methods and apparatus of the present invention than in the 
prior art. 

In the case of a ball grid array substrate, by way of example only, other 
areas among which it may be desirable to distinguish or characterize, using the 

10 apparatus and methods of the present invention, include: a bare substrate, a metal 
coating such as gold plating over copper pads and connectors; the existence of an at 
least partially transparent coating, such as a solder mask, covering copper connectors, 
gold plated connectors or substrate; different substances such as copper connector or 
gold plated connectors lying under an at least partially transparent coating such as a 

15 solder mask; surface residues such as, for example, a photoresist residue; voids in a 
metal coating; surface irregularities; scratches; foreign materials; stains; and oxidation 
of copper connectors or gold plating. It may also be desirable to distinguish the shape of 
an area such as a metal connector, a metal coating or a surface mark. The examples 
given immediately above are intended to be by way of example and not to be limiting, it 

20 being appreciated that it may be desirable to distinguish or characterize many kinds of 
areas. It is believed that the apparatus and methods which form the current invention are 
sufficiently flexible to enable inspection and image analysis for many different materials 
and materials combinations on objects, such as BGAs, under inspection. 

It is appreciated that different regions in BGAs, as referred to above, will 

25 generally have different colors, color populations or other optical characteristics as 
described below, and that different areas may therefore be distinguished by 
distinguishing the colors and optical characteristics thereof. It is particularly noted in 
this regard that different substances, including different metals, and materials lying 
under a partially transparent coating, such as a solder mask, may generally be 

30 distinguished and classified, using the present invention, accordance with the different 
colors and other optical characteristics thereof. 



4 



WO 01/07893 



PCT/IL00/00434 



The present invention also seeks to provide improvements in processing 
of monochrome images, especially in applications similar to those mentioned above 
with respect to color images. 

There is thus provided in accordance with a preferred embodiment of the 
present invention a method for determining a location of a border in a color image, the 
image including at least two color populations, between a first color region associated 
with a first one of the two color populations and a second color region associated with a 
second one of the two color populations, both the first color region and the second color 
region being included in the color image, the method including identifying an 
approximate border location between the first color region and the second color region, 
determining a plurality of candidate border locations between the first color region and 
the second color region, each of the plurality of candidate border locations being 
determined by applying a corresponding border location method chosen from among a 
plurality of border location methods, choosing one method from among the plurality of 
border location methods as a preferred method, and determining a location of a border 
between the first color region and the second color region by designating one of the 
plurality of candidate border locations associated with the preferred method as the 
border. 

Further in accordance with a preferred embodiment of the present 
invention the color image includes an image of a patterned article. 

Still further in accordance with a preferred embodiment of the present 
invention the color image includes a color image of an electrical circuit. 

Additionally in accordance with a preferred embodiment of the present 
invention the color image includes a color image of a lead frame. 

Moreover in accordance with a preferred embodiment of the present 
invention the preferred border location method is chosen based on a rule for choosing a 
preferred border location method for a predetermined combination of color populations. 

Further in accordance with a preferred embodiment of the present 
invention the preferred border location method is chosen based on comparison of the 
border location result from applying at least two border location methods. 
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There is also provided in accordance with another preferred embodiment 
of the present invention a method for determining a location of a border in a color 
image, the image including at least two color populations, between a first color region 
associated with a first one of the two color populations and a second color region 
associated with a second one of the two color populations, both the first color region and 
the second color region being included in the color image, the method including 
providing a border analysis map including, for each distinct pair of color populations, an 
indication of a preferred method for identifying the location of a border between the 
color populations, identifying an approximate border location between the first color 
region and the second color region, and determining a location of a border between the 
first color region and the second color region by using the preferred method indicated, in 
the border analysis map, for identifying the location of a border between the first color 
population and the second color population. 

Further in accordance with a preferred embodiment of the present 
invention the color image includes an image of a patterned article. 

Still further in accordance with a preferred embodiment of the present 
invention the color image includes a color image of an electrical circuit. 

Additionally in accordance with a preferred embodiment of the present 
invention the color image includes a color image of a lead frame. 

Moreover in accordance with a preferred embodiment of the present 
invention the electrical circuit includes a printed circuit board. 

Further in accordance with a preferred embodiment of the present 
invention the color image includes a color image of an object, and each color population 
is associated with one material of a plurality of available materials 

Still further in accordance with a preferred embodiment of the present 
invention the color image also includes a third color region associated with a third color 
population, and the identifying step includes, for each two color regions among the first 
color region, the second color region, and the third color region which border each 
other, identifying an approximate border location between the two color regions, and the 
determining step includes, for each the two color regions, determining a location of a 
border between the two color regions by using the preferred method indicated, in the 
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border analysis map, for identifying the location of a border between the two color 
populations associated with the two color regions. 

Additionally in accordance with a preferred embodiment of the present 
invention the method also includes defining a window having a window size, examining 
a portion of the color image falling within the window to determine the number of 
distinct color regions at least a part of which are included in the portion, and if the 
number of distinct color regions is greater than two, classifying all borders within the 
window as belonging to a junction and determining a location of each border within the 
window using a preferred method for determining the location of a border belonging to 
a junction. 

Moreover in accordance with a preferred embodiment of the present 
invention the preferred method for determining the location of a border belonging to a 
junction includes a high- water-mark method. 

Further in accordance with a preferred embodiment of the present 
invention the color image is represented using a color image representation method 
having a plurality of color defining characteristics, and each preferred method is selected 
from a plurality of available methods, and the plurality of available methods includes, 
for each color defining characteristic within the plurality of color defining 
characteristics, a method including performing an edge locating method in a component 
of the color image associated with the color defining characteristic. 

Still further in accordance with a preferred embodiment of the present 
invention the performing an edge locating method in a component includes performing a 
sub-pixel contour-element locating method. 

Additionally in accordance with a preferred embodiment of the present 
invention the performing an edge locating method only in a component includes 
performing a high- water-mark method. 

Moreover in accordance with a preferred embodiment of the present 
invention the color image representation method includes an RGB method and the 
plurality of color defining characteristics includes an R component, a G component, and 
a B component. 
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Further in accordance with a preferred embodiment of the present 
invention the color image representation method includes an HSI method and the 
plurality of color defining characteristics includes an H component, and S component, 
and an I component. 

There is also provided in accordance with another preferred embodiment 
of the present invention a method for producing a color morphology map from a color 
image, the method including providing a color image, reducing the color image to 
produce a reduced image including a plurality of color populations, determining borders 
between adjacent ones of the plurality of color populations in the reduced image, 
segmenting the reduced image to produce a binary image including borders and at least 
one non-border region, producing a skeleton image of the at least one non-border region, 
the skeleton image including a plurality of portions, and assigning to each portion of the 
skeleton a color identity corresponding to one of the color populations. 

Further in accordance with a preferred embodiment of the present 
invention the assigning step includes assigning a color population as a function of the 
spatial location of a portion of the skeleton relative to the color population. 

Still further in accordance with a preferred embodiment of the present 
invention the assigning step further includes superimposing the skeleton image on the 
reduced image. 

Additionally in accordance with a preferred embodiment of the present 
invention the color image includes a 24 bit color image. 

Moreover in accordance with a preferred embodiment of the present 
invention the reduced image includes a 3 bit image, and the plurality of color 
populations includes at most 8 color populations. 

Further in accordance with a preferred embodiment of the present 
invention the determining step includes determining borders to an accuracy of one pixel. 

Still further in accordance with a preferred embodiment of the present 
invention the method also includes identifying the presence of at least one 
morphological feature based, at least in part, on a result of the method. 

Additionally in accordance with a preferred embodiment of the present 
invention the color image includes an image of an patterned article. 
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Moreover in accordance with a preferred embodiment of the present 
invention the patterned article includes an electrical circuit. 

There is also provided in accordance with another preferred embodiment 
of the present invention a method for determining color contour elements (eels) in a 
color image, the method including providing a color image, reducing the color image to 
produce a reduced image including a plurality of color populations, determining borders 
between adjacent ones of the plurality of color populations, the borders including a 
plurality of border segments each border segment being surrounded by two color 
populations, and for each border segment, assigning a value to the border segment 
based, at least in part, on the two color populations surrounding the border segment. 

Further in accordance with a preferred embodiment of the present 
invention the color image includes a 24 bit color image. 

Still further in accordance with a preferred embodiment of the present 
invention the reduced image includes a 3 bit image. 

Additionally in accordance with a preferred embodiment of the present 
invention the determining step includes determining borders to an accuracy of one pixel. 

Moreover in accordance with a preferred embodiment of the present 
invention the determining step includes determining borders to an accuracy of less than 
one pixel. 

Further in accordance with a preferred embodiment of the present 
invention each border segment has a first side and a second side, and the assigning step 
includes assigning a value based, at least in part, on the color population on the first side 
of the border segment and on the color population on the second side of the border 
segment. 

Still further in accordance with a preferred embodiment of the present 
invention the color image includes an image of an patterned article. 

Additionally in accordance with a preferred embodiment of the present 
invention the patterned article includes an electrical circuit. 

There is also provided in accordance with another preferred embodiment 
of the present invention a method for identifying irregularities in a smooth curve in an 
image, the method including identifying an ordered plurality of points along the smooth 
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curve, each of the ordered plurality of points having at least one neighboring point in the 
ordered plurality of points, determining a normal direction to the smooth curve at each 
of the ordered plurality of points, and identifying a point of the ordered plurality of 
points as being part of an irregularity based, at least in part, on a local normal difference 
5 including a difference between the normal direction to the smooth curve at the point and 
a normal direction to the smooth curve at the at least one neighboring point of the point. 

Further in accordance with a preferred embodiment of the present 
invention the image includes an image of a patterned article. 

Still further in accordance with a preferred embodiment of the present 
1 0 invention the image includes an image of an electrical circuit. 

Additionally in accordance with a preferred embodiment of the present 
invention the local normal difference includes a difference in direction of more than a 
predetermined angle. 

Moreover in accordance with a preferred embodiment of the present 
15 invention the predetermined angle includes an angle of approximately 22 degrees. 

Further in accordance with a preferred embodiment of the present 
invention the image includes a multiplicity of pixels and each of the ordered plurality of 
points is associated with one pixel. 

Still further in accordance with a preferred embodiment of the present 
20 invention each pixel included in the smooth curve is associated with one of the ordered 
plurality of points. 

Additionally in accordance with a preferred embodiment of the present 
invention only some pixels included in the smooth curve are associated with one of the 
ordered plurality of points. 
25 Moreover in accordance with a preferred embodiment of the present 

invention the some pixels are chosen so that a fixed number of pixels not associated 
with one of the ordered plurality of points falls between each one of the ordered 
plurality of points and each the at least one neighbor of each one of the ordered plurality 
of points. 

30 Further in accordance with a preferred embodiment of the present 

. invention the method also includes determining a feature depth and feature aperture for 
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the irregularity, and classifying the feature based, at least in part, on a result of the 
determining step. 

Still further in accordance with a preferred embodiment of the present 
invention the classifying includes classifying into one of a predetermined set of 
5 categories including at least one of the following categories: a nick, a protrusion, and a 
contour. 

Additionally in accordance with a preferred embodiment of the present 
invention the method includes a method for detecting an irregularity in an electrical 
circuit, the method including identifying an irregularity in a smooth curve representing a 
1 0 border between two materials in an image of the electrical circuit in accordance with the 
method. 

There is also provided in accordance with another preferred embodiment 
of the present invention a method for classifying an irregularity in an image of an object, 
the method including determining a feature depth and feature aperture for the 
15 irregularity, and classifying the feature based, at least in part, on a result of the 
determining step. 

Further in accordance with a preferred embodiment of the present 
invention the object includes an electrical circuit. 

Still further in accordance with a preferred embodiment of the present 
20 invention the classifying includes classifying into one of a predetermined set of 
categories including at least one of the following categories: a nick, a protrusion, and a 
contour. 

There is also provided in accordance with another preferred embodiment 
of the present invention a method for identifying a defect in a portion of an image of an 
25 object, the portion being close to a location in the image identified as an edge, the 
method including identifying an excluded direction associated with the edge, and 
searching for defects, close to the location identified as an edge, in a plurality of 
directions not including the excluded direction. 

Further in accordance with a preferred embodiment of the present 
30 invention the object includes an electrical circuit. 
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Still further in accordance with a preferred embodiment of the present 
invention the excluded direction includes a direction perpendicular to the edge. 

Additionally in accordance with a preferred embodiment of the present 
invention the defect includes a scratch. 

Moreover in accordance with a preferred embodiment of the present 
invention the defect includes a stain. 

There is also provided in accordance with another preferred embodiment 
of the present invention a method for identifying a surface defect in an image of an 
object, the image including a multiplicity of pixels, the method including choosing a 
region in which a surface defect is to be identified, subsampling the region thereby 
producing a subsampled image of the region, analyzing the subsampled image and 
identifying therefrom a surface defect in the region. 

Further in accordance with a preferred embodiment of the present 
invention the image includes a color image of an electrical circuit. 

Still further in accordance with a preferred embodiment of the present 
invention the subsampling includes subsampling non-adjacent pixels. 

Additionally in accordance with a preferred embodiment of the present 
invention separation of non-adjacent pixels respective of adjacent pixels includes a 
subsampling ratio. 

Moreover in accordance with a preferred embodiment of the present 
invention the subsampling ratio is between approximately 1 13 and approximately 149. 

Further in accordance with a preferred embodiment of the present 
invention the subsampling includes subsampling in accordance with an approximate 
average distance between subsampled pixels. 

Still further in accordance with a preferred embodiment of the present 
invention the approximate average distance between subsampled pixels is between 
approximately 5 pixels and approximately 1 1 pixels. 

Additionally in accordance with a preferred embodiment of the present 
invention the subsampling step includes subsampling a portion of the region, and the 
subsampling step is performed a plurality of times using a plurality of portions of the 
region. 

12 
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Moreover in accordance with a preferred embodiment of the present 
invention at least two of the plurality of portions of the region include partially 
overlapping portions of the region. 

There is also provided in accordance with another preferred embodiment 
5 of the present invention a method for applying an image processing operator to a pixel 
in an array of pixels to produce a value, the method including providing an n x n spread 
of a central pixel, if the central pixel includes an edge pixel, assigning the value 0 to a 
result, if the central pixel does not include an edge pixel, performing the following steps 
for each one pixel in the n x n spread if the one pixel is not an edge pixel, choosing the 

10 one pixel as a chosen pixel, if the one pixel is an edge pixel and a mirror image of the 
one pixel within the n x n spread is not an edge pixel, choosing the mirror image as the 
chosen pixel, otherwise choosing the central pixel as the chosen pixel, and adding to the 
sum a product of the value of the chosen pixel and the result of applying a Gaussian 
operator to the central pixel, and dividing the sum by the square of (n+1) to produce a 

15 result. 

Further in accordance with a preferred embodiment of the present 
invention n is equal to 3. 

There is also provided in accordance with another preferred embodiment 
of the present invention a method for computing a gradient with a convoluted operator 

20 including providing an image processing operator in the form of an n x n array of 
values, wherein n is an odd integer and the central value of the n x n array may take on 
either a positive value or a negative value, applying the image processing operator with 
the central value having a positive value to produce a first intermediate result, applying 
the image processing operator with the central value having a negative value to produce 

25 a second intermediate result, and summing the first intermediate result and the second 
intermediate result to produce a final result. 

Further in accordance with a preferred embodiment of the present 
invention the method also includes comparing the arithmetic sign of the first 
intermediate result and the arithmetic sign of the second intermediate result, and 

30 determining the presence of a local extremum based on a result of the comparing step. 
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Still further in accordance with a preferred embodiment of the present 
invention the determining includes determining with less than complete certainty. 

Additionally in accordance with a preferred embodiment of the present 
invention n is equal to 5. 

There is also provided in accordance with another preferred embodiment 
of the present invention an automated optical inspection device suitable for inspection 
of patterned articles including at least one detector providing a polychromatic image 
output of at least a portion of a patterned article having a plurality of elements, and 
processing circuitry receiving the polychromatic image output and providing at least one 
of the following inspection functionalities inspection of a metal coating on at least one 
of the elements in the plurality of elements, and inspection of an at least partially 
transparent coating on at least one element in the plurality of elements. 

There is also provided in accordance with still another preferred 
embodiment of the present invention an automated optical inspection device suitable for 
inspection of patterned articles including at least one detector providing a polychromatic 
image output of at least a portion of a patterned article including a plurality of elements, 
and processing circuitry receiving the polychromatic image output and providing the 
following inspection functionalities: inspection of a metal coating on at least one 
element in the plurality of elements, and inspection of an at least partially transparent 
coating on at least one element in the plurality of elements. 

Further in accordance with a preferred embodiment of the present 
invention the processing circuitry receiving the polychromatic image output provides the 
following additional inspection functionality: detection of residues on a surface of the 
patterned article. 

Still further in accordance with a preferred embodiment of the present 
invention the processing circuitry receiving the polychromatic image output provides the 
following additional inspection functionality: detection of residues on a surface of the 
patterned article. 

Additionally in accordance with a preferred embodiment of the present 
invention the functionality of inspection of a metal coating on at least one of the 
elements in the plurality of elements includes at least one of the following inspection 
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modalities: detection of voids in the metal coating, detection of surface irregularities in 
the metal coating, detection of scratches in the metal coating, detection of the presence 
of foreign materials on the metal coating, detection of stains on the metal coating, 
detection of oxidation of the metal coating, and detection of the shape of the metal 
5 coating. 

Moreover in accordance with a preferred embodiment of the present 
invention the functionality of inspection of a metal coating on at least one of the 
elements in the plurality of elements includes at least one of the following inspection 
modalities: detection of voids in the metal coating, detection of surface irregularities in 
10 the metal coating, detection of scratches in the metal coating, detection of the presence 
of foreign materials on the metal coating, detection of stains on the metal coating, 
detection of oxidation of the metal coating, and detection of the shape of the metal 
coating. 

Further in accordance with a preferred embodiment of the present 

1 5 invention the functionality of inspection of an at least partially transparent coating on at 
least one of the elements in the plurality of elements includes at least one of the 
following inspection modalities: detection of voids in the at least partially transparent 
coating, detection of surface irregularities in the at least partially transparent coating, 
detection of scratches in the at least partially transparent coating, detection of the 

20 presence of foreign materials on the at least partially transparent coating, detection of 
stains on the at least partially transparent coating, and detection of the shape of the at 
least partially transparent coating. 

Still further in accordance with a preferred embodiment of the present 
invention the functionality of inspection of an at least partially transparent coating on at 

25 least one of the elements in the plurality of elements includes at least one of the 
following inspection modalities: detection of voids in the at least partially transparent 
coating, detection of surface irregularities in the at least partially transparent coating, 
detection of scratches in the at least partially transparent coating, detection of the 
presence of foreign materials on the at least partially transparent coating, detection of 

30 stains on the at least partially transparent coating, and detection of the shape of the at 
least partially transparent coating. 
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Additionally in accordance with a preferred embodiment of the present 
invention the functionality of detection of residues on a surface of the patterned article 
includes at least one of the following detection modalities: detection of surface 
irregularities on a surface of the patterned article, detection of the presence of foreign 
materials on a surface of the patterned article, detection of stains on a surface of the 
patterned article, and detection of the shape of marks on a surface of the patterned 
article. 

Moreover in accordance with a preferred embodiment of the present 
invention the functionality of detection of residues on a surface of the patterned article 
includes at least one of the following detection modalities: detection of surface 
irregularities on a surface of the patterned article, detection of the presence of foreign 
materials on a surface of the patterned article, detection of stains on a surface of the 
patterned article, and detection of the shape of marks on a surface of the patterned 
article. 

Further in accordance with a preferred embodiment of the present 
invention the patterned article includes an electrical circuit. 

Still further in accordance with a preferred embodiment of the present 
invention the elements include conductors in the electrical circuit. 

Additionally in accordance with a preferred embodiment of the present 
invention the electrical circuit includes a printed circuit board. 

Moreover in accordance with a preferred embodiment of the present 
invention the electrical circuit includes a ball grid array substrate. 

Further in accordance with a preferred embodiment of the present 
invention the patterned article comprises a lead frame. 

Still further in accordance with a preferred embodiment of the present 
invention the metal coating includes a metal plating. 

Additionally in accordance with a preferred embodiment of the present 
invention the elements include balls in the ball grid array substrate. 

Moreover in accordance with a preferred embodiment of the present 
invention the patterned article includes an etched metal substrate. 
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Further in accordance with a preferred embodiment of the present 
invention the patterned article includes an engraved metal substrate. 

There is also provided in accordance with another preferred embodiment 
of the present invention an automated optical inspection device suitable for inspection 
5 of an article and including at least one detector providing a polychromatic image output 
of at least a portion of an article, and processing circuitry receiving the polychromatic 
image output and providing edge detection to sub-pixel accuracy operative to 
distinguish borders among regions of different color. 

Further in accordance with a preferred embodiment of the present 
10 invention the processing circuitry is operative to distinguish a border between a first 
color region associated with a first color population and a second color region associated 
with a second color population by identifying an approximate border location between 
the first color region and the second color region and determining a location of a border 
between the first color region and the second color region by using a preferred method 
1 5 for identifying the location of a border between the first color population and the second 
color population. 

Still further in accordance with a preferred embodiment of the present 
invention the polychromatic image output includes a color image represented using a 
color image representation method having a plurality of color defining characteristics, 
20 and each preferred method is selected from a plurality of available methods, and the 
plurality of available methods includes, for each color defining characteristic within the 
plurality of color defining characteristics, a method including performing an edge 
locating method in a component of the color image associated with the color defining 
characteristic. 

25 Additionally in accordance with a preferred embodiment of the present 

invention the edge locating method includes a sub-pixel contour-element locating 
method. 

There is also provided in accordance with another preferred embodiment 
of the present invention an automated optical inspection device suitable for inspection 
30 of an article and including at least one detector providing an image output of at least a 
portion of an article having at least three different regions, each of which regions is 
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distinguished at least by an optical characteristic detectable by the detector, and 
processing circuitry receiving the image output and providing edge detection to 
sub-pixel accuracy operative to distinguish borders of at least two of the at least three 
different regions. 

5 Further in accordance with a preferred embodiment of the present 

invention the processing circuitry receives the image output and provides edge detection 
to sub-pixel accuracy operative to distinguish borders of the at least three different 
regions. 

Still further in accordance with a preferred embodiment of the present 
10 invention the processing circuitry provides mapping based at least partially on the image 
output which identifies materials in each of the different regions having a common 
border. 

There is also provided in accordance with another preferred embodiment 
of the present invention an automated optical inspection device suitable for inspection 

15 of an article and including at least one detector providing an image output of at least a 
portion of an article, and processing circuitry receiving the image output and providing 
mapping based at least partially on the image output which identifies materials in each 
of different regions of the at least a portion of the article having a common border. 

Further in accordance with a preferred embodiment of the present 

20 invention the processing circuitry is operative to sub-pixel accuracy. 

There is also provided in accordance with another preferred embodiment 
of the present invention an automated optical inspection device suitable for inspection 
of an article including at least one detector providing a polychromatic image output of at 
least a portion of an article, processing circuitry receiving the polychromatic image 

25 output and providing at least one of the following inspection functionalities 
segmentation of the polychromatic image into a segmented color image including a 
plurality of color regions each having one of a plurality of representative color 
possibilities, binarization of the segmented color image into a binary image including 
regions and borders, morphological erosion of the regions to provide a skeleton 

30 representing the regions, and assignment of color regions to elements in the skeleton. 
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Further in accordance with a preferred embodiment of the present 
invention the processing circuitry is operative to superimpose the skeleton and the 
segmented color image, and to assign data representative of regions to elements in the 
skeleton as a function of the superimposition. 

Still further in accordance with a preferred embodiment of the present 
invention the processing circuitry is operative to assign data representative of regions to 
elements in the skeleton the superimposed location of an element of the skeleton in a 
color region. 

There is also provided in accordance with another preferred embodiment 
of the present invention an automated optical inspection device suitable for inspection 
of an article including at least one detector providing a polychromatic image output of at 
least a portion of an article, and processing circuitry receiving the polychromatic image 
output and operative to segment the polychromatic image into plurality of color regions 
chosen from among a predefined number of color region possibilities, and to generate a 
representation of the polychromatic image including color region data representing each 
color region in the image, and border data for representing each border among color 
regions. 

There is also provided in accordance with still another preferred 
embodiment of the present invention a color image representing a patterned article, the 
color image including a morphologically eroded polychromatic image of the patterned 
article. 

Further in accordance with a preferred embodiment of the present 
invention colors of portions of the morphologically eroded polychromatic image 
indicate materials at various regions of the patterned article. 

Still further in accordance with a preferred embodiment of the present 
invention the patterned article includes an electrical circuit. 

Additionally in accordance with a preferred embodiment of the present 
invention the elements include conductors in the electrical circuit. 

Moreover in accordance with a preferred embodiment of the present 
invention the electrical circuit includes a printed circuit board. 
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Further in accordance with a preferred embodiment of the present 
invention the electrical circuit includes a ball grid array substrate. 

Still further in accordance with a preferred embodiment of the present 
invention the patterned article includes a lead frame. 
5 Additionally in accordance with a preferred embodiment of the present 

invention the metal coating includes a metal plating. 

Moreover in accordance with a preferred embodiment of the present 
invention the elements include balls in the ball grid array substrate. 

Further in accordance with a preferred embodiment of the present 
1 0 invention the patterned article includes an etched metal substrate. 

Still further in accordance with a preferred embodiment of the present 
invention the patterned article includes an engraved metal substrate. 

There is also provided in accordance with another preferred embodiment 
of the present invention an automated optical inspection device suitable for inspection 
15 ot an article including at least one detector providing a polychromatic image output of at 
least a portion of an article, and processing circuitry receiving the polychromatic image 
output and providing a color image representing the article, the color image including a 
morphologically eroded polychromatic image of at least part of the article. 

Further in accordance with a preferred embodiment of the present 
20 invention colors of portions of the morphologically eroded polychromatic image 
indicate materials at various regions of the article. 

There is also provided in accordance with another preferred embodiment 
of the present invention a color image representing a patterned article, the color image 
including a map which indicates borders between different regions of an article and 
25 identifies materials in each of the different regions having a common border. 

Further in accordance with a preferred embodiment of the present 
invention colors of portions of the map indicate materials at the different regions. 

Still further in accordance with a preferred embodiment of the present 
invention each color includes one color from a predetermined possibility of colors. 
30 Additionally in accordance with a preferred embodiment of the present 

invention the patterned article includes an electrical circuit. 
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Moreover in accordance with a preferred embodiment of the present 
invention the elements include conductors in the electrical circuit. 

Further in accordance with a preferred embodiment of the present 
invention the electrical circuit includes a printed circuit board. 
5 Still further in accordance with a preferred embodiment of the present 

invention the electrical circuit includes a ball grid array substrate. 

Additionally in accordance with a preferred embodiment of the present 
invention the patterned article includes a lead frame. 

Moreover in accordance with a preferred embodiment of the present 
1 0 invention the metal coating includes a metal plating. 

Further in accordance with a preferred embodiment of the present 
invention the elements include balls in the ball grid array substrate. 

Still further in accordance with a preferred embodiment of the present 
invention the patterned article includes an etched metal substrate. 
1 5 Additionally in accordance with a preferred embodiment of the present 

invention the patterned article includes an engraved metal substrate. 

There is also provided in accordance with another preferred embodiment 
of the present invention an automated optical inspection device suitable for inspection 
of an article including at least one detector providing a polychromatic image output of at 
20 least a portion of an article, and processing circuitry receiving the polychromatic image 
output and providing a color image representing the article, the color image including a 
map which indicates borders between different regions of an article and identifies 
materials in each of the different regions having a common border. 

Further in accordance with a preferred embodiment of the present 
25 invention colors of portions of the map indicate materials at various regions of the 
article. 

Still further in accordance with a preferred embodiment of the present 
invention the processing circuitry receives the image output and provides border 
indication to sub-pixel accuracy. 
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Additionally in accordance with a preferred embodiment of the present 
invention each color includes a color chosen from among a predetermined possibility of 
colors. 

Moreover in accordance with a preferred embodiment of the present 
5 invention the patterned article includes an electrical circuit. 

Further in accordance with a preferred embodiment of the present 
invention the electrical circuit includes a printed circuit board. 

Still further in accordance with a preferred embodiment of the present 
invention the electrical circuit includes a ball grid array substrate. 
10 Additionally in accordance with a preferred embodiment of the present 

invention the patterned article includes a lead frame. 

Moreover in accordance with a preferred embodiment of the present 
invention the different materials include different metals. 

Further in accordance with a preferred embodiment of the present 
15 invention the different regions include uncoated metal conductors, coated metal 
conductors, uncoated substrate, and coated substrate. 

Still further in accordance with a preferred embodiment of the present 
invention the patterned article includes an etched metal substrate. 

Additionally in accordance with a preferred embodiment of the present 
20 invention the patterned article includes an engraved metal substrate. 

There is also provided in accordance with another preferred embodiment 
of the present invention an automated optical inspection device suitable for inspection 
of a patterned article including at least one detector providing an image output of at least 
a portion of a patterned article including at least three materials having detectable edges, 
25 and processing circuitry receiving the image output and providing an inspection 
functionality including binarization of the image output to provide a binary image 
including region image data and border image data. 

Further in accordance with a preferred embodiment of the present 
invention the patterned article includes an electrical circuit. 
30 Still further in accordance with a preferred embodiment of the present 

invention the electrical circuit includes a printed circuit board. 
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Additionally in accordance with a preferred embodiment of the present 
invention the printed circuit board includes a ball grid array substrate. 

Moreover in accordance with a preferred embodiment of the present 
invention the patterned article includes a lead frame. 

Further in accordance with a preferred embodiment of the present 
invention the patterned article includes an etched metal substrate. 

Still further in accordance with a preferred embodiment of the present 
invention the patterned article includes an engraved metal substrate. 

There is also provided in accordance with another preferred embodiment 
of the present invention an automated optical inspection device suitable for inspection 
of a patterned article including at least one detector providing an image output of at least 
a portion of a patterned article including a plurality of materials having detectable 
borders between the plurality of materials, and processing circuitry receiving the image 
output and providing an inspection functionality including pattern analysis of image data 
relating to the borders to provide an indication of the presence of nicks and protrusions 
along the borders. 

Further in accordance with a preferred embodiment of the present 
invention the patterned article includes an electrical circuit. 

Still further in accordance with a preferred embodiment of the present 
invention the patterned article includes a printed circuit board. 

Additionally in accordance with a preferred embodiment of the present 
invention the printed circuit board includes a ball grid array substrate. 

Moreover in accordance with a preferred embodiment of the present 
invention the patterned article includes a lead frame. 

Further in accordance with a preferred embodiment of the present 
invention the plurality of materials includes at least three materials. 

Still further in accordance with a preferred embodiment of the present 
invention the materials include materials from the following group uncoated metal 
conductors, coated material conductors, uncoated substrate, coated substrate. 

Additionally in accordance with a preferred embodiment of the present 
invention the patterned article includes an etched metal substrate. 
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Moreover in accordance with a preferred embodiment of the present 
invention the patterned article includes an engraved metal substrate. 

Further in accordance with a preferred embodiment of the present 
invention wherein the inspection functionality including pattern analysis includes 
identifying an ordered plurality of points along a border defined by the image data 
relating to the borders, each of the ordered plurality of points having at least one 
neighboring point in the ordered plurality of points, determining a normal direction to 
the smooth curve at each of the ordered plurality of points, and identifying a point of the 
ordered plurality of points as being part of an irregularity including one of a nick and a 
protrusion based, at least in part, on a local normal difference including a difference 
between the normal direction to the smooth curve at the point and a normal direction to 
the smooth curve at the at least one neighboring point of the point. 

There is also provided in accordance with another preferred embodiment 
of the present invention an automated optical inspection device suitable for inspection 
of an article including at least one detector providing an image output of at least a 
portion of an article including multiple materials having detectable borders between the 
materials, and processing circuitry receiving the image output and providing at least one 
of the following inspection functionalities: segmentation of the image output into a 
segmented image including region image data and border image data, analysis of the 
region image data using a first technique, and analysis of the border image data using a 
second technique, different from the first technique. 

Further in accordance with a preferred embodiment of the present 
invention the article includes an electrical circuit. 

Still further in accordance with a preferred embodiment of the present 
invention the first technique includes neighborhood analysis and the second technique 
includes border following analysis for differentiating border-near pixels from 
non-border-near pixels. 

There is also provided in accordance with another preferred embodiment 
of the present invention an automated optical inspection device suitable for inspection 
of an article including at least one detector providing a polychromatic image output of at 
least a portion of an article including multiple materials having detectable borders 
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between the materials, and processing circuitry receiving the polychromatic image 
output and providing at least one of the following inspection functionalities: 
segmentation of the polychromatic image output into a segmented color image including 
color regions chosen from among a set of predetermined colors, analysis of the region 
image data relating to a first color using a first technique, and analysis of the region 
image data relating to a second color using a second technique, different from the first 
technique. 

Further in accordance with a preferred embodiment of the present 
invention the article includes an electrical circuit. 

Still further in accordance with a preferred embodiment of the present 
invention the electrical circuit includes a printed circuit board. 

Additionally in accordance with a preferred embodiment of the present 
invention the printed circuit board includes a ball grid array substrate. 

Moreover in accordance with a preferred embodiment of the present 
invention the article includes a lead frame. 

There is also provided in accordance with another preferred embodiment 
of the present invention an automated optical inspection device suitable for inspection 
of an article including at least one detector providing an image output of at least a 
portion of an article including multiple materials having detectable borders between the 
materials, and processing circuitry receiving the image output and providing at least one 
of the following inspection functionalities: segmentation of the image output into a 
segmented image including region image data and border image data, and smoothing the 
region image data in a manner not influenced by at least some of the border image data. 

Further in accordance with a preferred embodiment of the present 
invention the article includes an electrical circuit. 

Still further in accordance with a preferred embodiment of the present 
invention the electrical circuit includes a printed circuit board. 

Additionally in accordance with a preferred embodiment of the present 
invention the printed circuit board includes a ball grid array substrate. 

Moreover in accordance with a preferred embodiment of the present 
invention the article includes a lead frame. 
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There is also provided in accordance with another preferred embodiment 
of the present invention an automated optical inspection device suitable for inspection 
of an electrical circuit including at least one detector providing an image output of at 
least a portion of an electrical circuit including multiple materials, and processing 
circuitry receiving the image output and providing an output indication of the presence 
of at least one predetermined material. 

Further in accordance with a preferred embodiment of the present 
invention the electrical circuit includes a printed circuit board. 

Still further in accordance with a preferred embodiment of the present 
invention the printed circuit board includes a ball grid array substrate. 

Additionally in accordance with a preferred embodiment of the present 
invention the electrical circuit includes a flat panel display substrate. 

Moreover in accordance with a preferred embodiment of the present 
invention the processing circuitry provides an output indication of the presence of at 
least two materials out of a group of at least three predetermined materials. 

Further in accordance with a preferred embodiment of the present 
invention the processing circuitry provides an output indication of the presence of at 
least three predetermined materials. 

Still further in accordance with a preferred embodiment of the present 
invention the processing circuitry provides an output indication of the presence of 
copper. 

Additionally in accordance with a preferred embodiment of the present 
invention the processing circuitry provides an output indication of the presence of gold. 

Moreover in accordance with a preferred embodiment of the present 
invention the processing circuitry provides an output indication of the presence of a 
solder mask material. 

Further in accordance with a preferred embodiment of the present 
invention the processing circuitry provides an output indication of the presence of silver. 

Still further in accordance with a preferred embodiment of the present 
invention the processing circuitry provides an output indication of the presence of 
photoresist residue. 
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Additionally in accordance with a preferred embodiment of the present 
invention the processing circuitry provides an output indication of the presence of 
oxidized metal. 

Moreover in accordance with a preferred embodiment of the present 
5 invention the image output represents a portion of the electrical circuit as viewed 
through a translucent overlay. 

There is also provided in accordance with another preferred embodiment 
of the present invention an automated optical inspection device suitable for inspection 
of a patterned article including at least one detector providing an image output of at least 
1 0 a portion of a patterned article through a translucent overlay at least partially covering 
the patterned article, and processing circuitry receiving the image output and providing 
an output indication which is independent of the translucent overlay. 

Further in accordance with a preferred embodiment of the present 
invention the patterned article includes an electrical circuit. 
15 Still further in accordance with a preferred embodiment of the present 

invention the electrical circuit includes a printed circuit board. 

Additionally in accordance with a preferred embodiment of the present 
invention the printed circuit board includes a ball grid array substrate. 

Moreover in accordance with a preferred embodiment of the present 
20 invention the processing circuitry is operative to provide an indication of parts of the 
pattern covered by the translucent overlay and parts of the pattern not covered by the 
translucent overlay. 

There is also provided in accordance with another preferred embodiment 
of the present invention apparatus for determining a location of a border in a color 
25 image, the image including at least two color populations, between a first color region 
associated with a first one of the two color populations and a second color region 
associated with a second one of the two color populations, both the first color region and 
the second color region being included in the color image, the apparatus including a 
border identification unit operative to identify an approximate border location between 
30 the first color region and the second color region, a candidate border determination unit 
operative to determine a plurality of candidate border locations between the first color 
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region and the second color region, each of the plurality of candidate border locations 
being determined by applying a corresponding border location method chosen from 
among a plurality of border location methods, a method selector operative to choose one 
method from among the plurality of border location methods as a preferred method, and 
5 a border determination unit operative to determine a location of a border between the 
first color region and the second color region by designating one of the plurality of 
candidate border locations associated with the preferred method as the border. 

There is also provided in accordance with still another preferred 
embodiment of the present invention apparatus for determining a location of a border in 

1 0 a color image, the image including at least two color populations, between a first color 
region associated with a first one of the two color populations and a second color region 
associated with a second one of the two color populations, both the first color region and 
the second color region being included in the color image, the apparatus including a 
border analysis map unit operative to provide a border analysis map including, for each 

1 5 distinct pair of color populations, an indication of a preferred method for identifying the 
location of a border between the color populations, a border identification unit operative 
to identify an approximate border location between the first color region and the second 
color region, and a border determination unit operative to determine a location of a 
border between the first color region and the second color region by using the preferred 

20 method indicated, in the border analysis map, for identifying the location of a border 
between the first color population and the second color population. 

There is also provided in accordance with yet another preferred 
embodiment of the present invention apparatus for producing a color morphology map 
from a color image, the apparatus including a reducing unit operative to reduce the color 

25 image to produce a reduced image including a plurality of color populations, a border 
determination unit operative to determine borders between adjacent ones of the plurality 
of color populations in the reduced image, a segmentation unit operative to segment the 
reduced image to produce a binary image including borders and at least one non-border 
region, a skeleton production unit operative to produce a skeleton image of the at least 

30 one non-border region, the skeleton image including a plurality of portions, and an 
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assigning unit operative to assign to each portion of the skeleton a color identity 
corresponding to one of the color populations. 

There is also provided in accordance with another preferred embodiment 
of the present invention apparatus for identifying irregularities in a smooth curve in an 
image, the apparatus including an identification unit operative to identify an ordered 
plurality of points along the smooth curve, each of the ordered plurality of points having 
at least one neighboring point in the ordered plurality of points, a normal direction unit 
operative to determine a normal direction to the smooth curve at each of the ordered 
plurality of points, and an irregularity unit operative to identify a point of the ordered 
plurality of points as being part of an irregularity based, at least in part, on a local 
normal difference including a difference between the normal direction to the smooth 
curve at the point and a normal direction to the smooth curve at the at least one 
neighboring point of the point. 

There is also provided in accordance with still another preferred 
embodiment of the present invention apparatus for classifying an irregularity in an 
image of an object, the apparatus including a dimension determination unit operative to 
determine a feature depth and feature aperture for the irregularity, and a classification 
unit operative to classify the feature based, at least in part, on an output of the dimension 
determination unit. 

There is also provided in accordance with yet another preferred 
embodiment of the present invention apparatus for identifying a defect in a portion of an 
image of an object, the portion being close to a location in the image identified as an 
edge, the apparatus including an excluded direction unit operative to identify an 
excluded direction associated with the edge, and a defects unit operative to search for 
defects, close to the location identified as an edge, in a plurality of directions not 
including the excluded direction. 

There is also provided in accordance with another preferred embodiment 
of the present invention apparatus for identifying a surface defect in a region of an 
image of an object, the image including a multiplicity of pixels, the apparatus including 
a subsampling unit operative to subsample the region in accordance with a subsampling 
ratio, thus producing a subsampled image of the region, a defect identification unit 
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operative to identify a surface defect in the subsampled image, and a correspondence 
unit operative to identify a surface defect in the region corresponding to the surface 
defect in the subsampled image. 

There is also provided in accordance with still another preferred 
5 embodiment of the present invention a method for inspection of patterned articles 
including providing a polychromatic image output of at least a portion of a patterned 
article having a plurality of elements, and receiving the polychromatic image output and 
providing at least one of the following inspection functionalities: inspection of a metal 
coating on at least one of the elements in the plurality of elements, and inspection of an 
1 0 at least partially transparent coating on at least one element in the plurality of elements. 

There is also provided in accordance with yet another preferred 
embodiment of the present invention a method for inspection of patterned articles 
including providing a polychromatic image output of at least a portion of a patterned 
article including a plurality of elements, and receiving the polychromatic image output 
15 and providing the following inspection functionalities inspection of a metal coating on 
at least one element in the plurality of elements, and inspection of an at least partially 
transparent coating on at least one element in the plurality of elements. 

There is also provided in accordance with another preferred embodiment 
of the present invention a method for inspection of an article including providing a 
20 polychromatic image output of at least a portion of an article, and receiving the 
polychromatic image output and providing edge detection to sub-pixel accuracy 
operative to distinguish borders among regions of different color. 

There is also provided in accordance with still another preferred 
embodiment of the present invention a method for inspection of an article including 
25 providing an image output of at least a portion of an article having at least three 
different regions, each of which regions is distinguished at least by an optical 
characteristic, and receiving the image output and providing edge detection to sub-pixel 
accuracy operative to distinguish borders of at least two of the at least three different 
regions. 

30 There is also provided in accordance with yet another preferred 

embodiment of the present invention a method for inspection of an article including 
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providing an image output of at least a portion of an article, and receiving the image 
output and providing mapping based at least partially on the image output which 
identifies materials in each of different regions of the at least a portion of the article 
having a common border. 
5 There is also provided in accordance with another preferred embodiment 

of the present invention a method for inspection of an article including providing a 
polychromatic image output of at least a portion of an article, and receiving the 
polychromatic image output and providing at least one of the following inspection 
functionalities segmentation of the polychromatic image into a segmented color image 
1 0 including a plurality of color regions each having one of a plurality of representative 
color possibilities, binarization of the segmented color image into a binary image 
including regions and borders, morphological erosion of the regions to provide a 
skeleton representing the regions, and assignment of color regions to elements in the 
skeleton. 

15 There is also provided in accordance with still another preferred 

embodiment of the present invention a method for inspection of an article including 
providing a polychromatic image output of at least a portion of an article, and receiving 
the polychromatic image output and providing a color image representing the article, the 
color image including a morphologically eroded polychromatic image of at least part of 

20 the article. 

There is also provided in accordance with yet another preferred 
embodiment of the present invention a method for inspection of an article including 
providing a polychromatic image output of at least a portion of an article, and receiving 
the polychromatic image output and providing a color image representing the article, the 
25 color image including a map which indicates borders between different regions of an 
article and identifies materials in each of the different regions having a common border. 

There is also provided in accordance with another preferred embodiment 
of the present invention a method for inspection of a patterned article including 
providing an image output of at least a portion of a patterned article including at least 
30 three materials having detectable edges, and receiving the image output and providing 
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an inspection functionality including binarization of the image output to provide a 
binary image including region image data and border image data. 

There is also provided in accordance with still another preferred 
embodiment of the present invention a method for inspection of a patterned article 
5 including providing an image output of at least a portion of a patterned article including 
a plurality of materials having detectable borders between the plurality of materials, and 
receiving the image output and providing an inspection functionality including pattern 
analysis of image data relating to the borders to provide an indication of the presence of 
nicks and protrusions along the borders. 

10 There is also provided in accordance with yet another preferred 

embodiment of the present invention a method for inspection of an article including 
providing an image output of at least a portion of an article including multiple materials 
having detectable borders between the materials, receiving the image output and 
providing at least one of the following inspection functionalities: segmentation of the 

15 image output into a segmented image including region image data and border image 
data, analysis of the region image data using a first technique, and analysis of the border 
image data using a second technique, different from the first technique. 

There is also provided in accordance with another preferred embodiment 
of the present invention a method for inspection of an article including providing a 

20 polychromatic image output of at least a portion of an article including multiple 
materials having detectable borders between the materials, and receiving the 
polychromatic image output and providing at least one of the following inspection 
functionalities: segmentation of the polychromatic image output into a segmented color 
image including color regions chosen from among a set of predetermined colors, 

25 analysis of the region image data relating to a first color using a first technique, and 
analysis of the region image data relating to a second color using a second technique, 
different from the first technique. 

There is also provided in accordance with still another preferred 
embodiment of the present invention a method for inspection of an article including 

30 providing an image output of at least a portion of an article including multiple materials 
having detectable borders between the materials, and receiving the image output and 
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providing at least one of the following inspection functionalities: segmentation of the 
image output into a segmented image including region image data and border image 
data, and smoothing the region image data in a manner not influenced by at least some 
of the border image data. 

There is also provided in accordance with yet another preferred 
embodiment of the present invention a method for inspection of an electrical circuit 
including providing an image output of at least a portion of an electrical circuit 
including multiple materials, and receiving the image output and providing an output 
indication of the presence of at least one predetermined material. 

There is also provided in accordance with another preferred embodiment 
of the present invention a method for inspection of a patterned article including 
providing an image output of at least a portion of a patterned article through a 
translucent overlay at least partially covering the patterned article, and receiving the 
image output and providing an output indication which is independent of the translucent 
overlay. 

There is also provided in accordance with still another preferred 
embodiment of the present invention a method for automatically optically inspecting 
color images ofj objects, including acquiring polychromatic images of an object, 
processing the polychromatic images and outputting a report indicating candidate 
defects and features chosen from among a predetermined set of features, and providing 
an image of a region surrounding each the candidate defect and feature. 

There is also provided in accordance with yet another preferred 
embodiment of the present invention apparatus for automatically optically inspecting 
color images of objects, including a sensor operative to acquire polychromatic images of 
an object, an image processor operative to process the polychromatic images and output 
a report indicating candidate defects and features chosen from among a predetermined 
set of features, and to provide an image of a region surrounding each the candidate 
defect and feature. 

Further in accordance with a preferred embodiment of the present 
invention the processor includes a binary image processor operative to process binary 
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images generated from the polychromatic image, and a color image processor operative 
to process the polychromatic images. 

Still further in accordance with a preferred embodiment of the present 
invention the apparatus also includes a morphological image processor operative to 
generate a skeleton map from a multicolor image derived from the polychromatic image. 

BRIEF DESCRIPTION OF THE DRAWINGS 
The present invention will be understood and appreciated more fully 
from the following detailed description, taken in conjunction with the drawings in 
which: 

Fig. 1 is a simplified block diagram illustration of an image analysis 
system constructed and operative in accordance with a preferred embodiment of the 
present invention; 

Fig. 2 is a simplified block diagram illustration of a preferred 
implementation of the material identification subunit 170 and material contours unit 180 
of Fig. I; 

Fig. 3 is a simplified flowchart illustration of a preferred method of a 
portion of the apparatus of Fig. 2; 

Fig. 4 is a simplified flowchart illustration of a preferred method of 
operation of the region expanding unit 173 of Fig. 2; 

Figs. 5 and 6 are simplified pictorial illustrations of a preferred 5x5 
large kernel and a preferred 3x3 corresponding small kernel, respectively; 

Fig. 7 is a simplified block diagram illustration of a preferred 
implementation of the contourization unit 175 of Fig. 2; 

Fig. 8 is a simplified flowchart illustration of a preferred method of 
operation of the best contour selector 178 of Fig. 7; 

Figs. 9 - 11 are simplified pictorial illustrations of preferred eel 
construction and nomenclature, useful in understanding the present invention; 

Fig. 12 is a simplified block diagram illustration of a preferred 
implementation of the color surface defects subunit 190 of Fig. 1; 
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Fig. 13 is a simplified flowchart illustration of a preferred method of 
operation of the colorl/color2 creation unit 400 of Fig. 12; 

Fig. 14 is a simplified block diagram illustration of a preferred 
implementation of a first portion of the directional enable images creator 402 of Fig. 12; 
5 Fig. 15 is a simplified pictorial illustration of an image pixel and 

neighboring virtual eel pixels, useful in understanding the operation of the apparatus of 
Fig. 14; 

Fig. 16 which is a simplified flowchart illustration of a preferred method 
of operation of the okdir map computation unit 416 of Fig. 14; 
10 Fig. 17 is a simplified flowchart illustration of a preferred 

implementation of step 418 of Fig. 16; 

Fig. 18 is a simplified flowchart illustration of a preferred 
implementation of step 419 of Fig. 16; 

Fig. 19 is a simplified pictorial illustration of an example of a process of 
1 5 erosion, useful in understanding the method of Fig. 1 8; 

Fig. 20 is a table showing okdir values, corresponding directions of 
measurement, and a preferred gradient operator for measuring, useful in understanding 
the method of Fig. 18; 

Fig. 21 is a simplified flowchart illustration of a preferred method of 
20 operation of a second portion of the directional enable images creator 402 of Fig. 12; 

Fig. 22 is a simplified pictorial illustration of a preferred choice of 
predetermined pixels, useful in understanding the method of Fig. 21; 

Fig. 23A is a table illustrating preferred heuristics for use in step 462 of 

Fig. 21; 

25 Fig. 23B is a simplified pictorial illustration showing a naming 

convention for the 9 pixels of the 3 x 3 virtual spread, useful in understanding Fig. 23 A; 

Fig. 24 is a simplified flowchart illustration of a preferred method of 
operation of a portion of the apparatus of Fig. 12; 

Fig. 25 is a simplified flowchart illustration of a preferred 
30 implementation of step 464 of Fig. 24; 
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Fig. 26 is a simplified flowchart illustration of a preferred 
implementation of a portion of step 470 of Fig. 24; 

Fig. 27 is a simplified flowchart illustration of a preferred method of 
operation of the line and space skeleton subunit 230 of Fig. 1; 

Figs. 28A and 28B are simplified pictorial illustrations of an array of 
pixels, before and after partial erosion, useful in understanding the method of Fig. 27; 

Fig. 29 is a simplified pictorial depiction of Euler's conditions, useful in 
understanding the method of Fig. 27; 

Figs. 3 OA - 30C are simplified pictorial illustrations useful in 
understanding a preferred method of operation of the morphology subunit 240 of Fig. 1; 

Figs. 31A - 31C are simplified pictorial illustrations of operators, useful 
in understanding Figs. 30A - 30C; 

Fig. 32 is a simplified flowchart illustration of a preferred method of 
color morphology analysis of an image; 

Figs. 33A and 33B, taken together, comprise a simplified flowchart 
illustration of a preferred method of operation of the nicks and protrusions subunit 220 
of Fig. 1; 

Fig. 34 is a simplified pictorial illustration helpful in understanding step 

890 of Fig. 33A; 

Fig. 35 is a simplified pictorial illustration of a portion of an image to be 
analyzed, useful for understanding the method of Figs. 33A and 33B; 

Fig. 36 is a simplified pictorial illustration of an array of virtual eel pixels 
useful in understanding the method of step 950 of Fig. 33B; 

Figs. 37A and 37B are simplified pictorial illustrations of examples of 

contours; 

Fig. 38 is a simplified pictorial illustration of a typical example of a 

protrusion; 

Figs. 39A and 39B are simplified pictorial illustrations of an array of 
virtual eel pixels, useful in understanding step 980 of Fig. 33B; 

Figs. 39C and 39D are simplified flowchart illustrations of methods 
useful in conjunction with the system of Fig. 1; and 
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Fig. 40 is a simplified pictorial illustration of an example of an electrical 
circuit, useful for understanding the operation of the present invention. 

DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT 
5 Overview 

Reference is now made to Fig. 1 which is a simplified block diagram 
illustration of an image analysis system 100 constructed and operative in accordance 
with a preferred embodiment of the present invention. It is appreciated that, while the 
system of Fig. 1 is particularly useful in analyzing an image which represents a 

10 patterned object, for example a BGA or other electric circuit, the system of Fig. 1 may 
generally be useful in image analysis, whether or not an image being analyzed represents 
a patterned object or any other object. Although reference in the present specification is 
made to BGAs for the purposes of illustrating the present invention, the term BGA as 
used in the present specification shall be deemed to refer to and additionally include 

1 5 printed circuit board substrates, laminated printed circuit boards, lead frames, flat panel 
displays, hybrid chip packaging substrates, tape automated bonding substrates, and any 
suitable patterned object including various etched and engraved metal substrates as may 
be used in medical implants. 

Image analysis system 100 is preferably included in an inspection system 

20 comprising illumination, image acquisition, object transportation, and software based 
image processing subsystems. Thus, in a case where an image being analyzed by the 
image analysis system 100 does comprise an image of an object, such as a patterned 
object like a BGA, it is appreciated that an appropriate image acquisition system (not 
shown) is typically employed to capture an image of the object, and that the image being 

25 analyzed by the image analysis system 100 is typically corrected in appropriate ways 
well-known in the art, including focus, compensation, resampling, and alignment. An 
appropriate image acquisition system suitable to provide an image to image analysis 
system 100 typically comprises at least one detector providing a polychromatic image 
output, as is well known in the art. The output of such an image acquisition system, or 

30 any similar image produced by other appropriate means, is generally termed herein a j 
"color image". 
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The input image to be analyzed typically comprises a color image in an 
appropriate color format, as is well known in the art. Preferably, the color image is 
provided in RGB (red-green-blue) format at high bit resolution, such as 24 -bit 
resolution. 

5 The image analysis system 1 00 is preferably operative to analyze an input 

image and to pass results of the analysis on for further processing, preferably a software 
image processing unit, or for reporting directly to an operator by way of an appropriate 
application interface. 

Preferably, as is well known in the art, the image analysis system 100 is 

10 implemented in custom hardware. While a software implementation of the image 
analysis system 100, to run on a general purpose computer, is believed to be possible 
and would, it is believed, be fully operative, a hardware implementation is preferred in 
order to achieve faster performance. For ease of description of the present invention, the 
methods are usually described so as to enable implementation of the principles of the 

1 5 present invention in either hardware of software. 

The image analysis system 1 00 preferably comprises a color processor, or 
"CLRP" unit 120, which preferably receives color information about the input image 
and is preferably operative, as described further below, to analyze the input image based 
on the received color information. 

20 The image analysis system 100 also preferably comprises a CABS unit 

130, which preferably receives monochrome, also known as gray level, information 
about the input image and is preferably operative, as described further below, to analyze 
the input image based on the received gray level information. The monochrome 
information may comprise information in one color, typically but not necessarily the 

25 color red, and typically comprises a high number of bits, such as 8 bits, of gray level 
information. A preferred implementation of the CABS unit 130 is described below with 
reference to Figs. 33A and 33B. 

The image analysis system 100 also preferably comprises a SKFD unit 
140, which preferably receives as input an output, described further below and typically 

30 comprising binarization information, from the CABS unit 130, and color information 
from the CLRP unit 120. The SKFD unit 140 is preferably operative, as described 
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further below, to analyze the input thereto and to produce therefrom characteristic 
information typically including a skeleton representation of the input image and 
morphology features found in the input image. 

The image analysis system 100 also preferably comprises a 
synchronization unit 150. Preferably and in order to obtain faster performance of the 
image analysis system 100, the CLRP unit 120, the CABS unit 130, and the SKFD unit 
140 operate in parallel. The synchronization unit 150 is preferably operatively 
associated with each of the CLRP unit 120, the CABS unit 130, and the SKFD unit 140 
and preferably receives output from each of the CLRP unit 120, the CABS unit 130, and 
the SKFD unit 140. Using any appropriate synchronization methods, such as 
hardware- based delay and synchronization methods which are well-known in the art, the 
synchronization unit 150 is preferably operative to mutually synchronize each of the 
inputs received thereat and to pass the synchronized inputs along as an output of the 
image analysis system 1 00. 

Preferably the image analysis system 1 00 generates a number of reports, 
which include information about various features and defects detected on an object 
being inspected, such as Binary Cels, Color Cels, Morphology Skeleton, Skeleton 
Features, Nick/Protrusion, Gray Surface Defects, Color Surface Defects, Defective 
Color Cels and Snapshot Images of Regions of Interest. These reports are inputted to a 
Software Image Processing device as described in detail in the Applicant's copending 
Israel Patent Application No. 131,282. In addition the image analysis system 100 
generated snapshot images of each feature and defect, as well as snapshot defects of 
particular regions of interest on the article as are predefined by a user prior to inspection. 
Typically a snapshot image is a polychromatic image of a region surrounding each 
feature, defect and predefined region of interest. 

Preferably, the snapshot image output of the image analysis system 100 
may be post-processed by a post-processing subsystem (not shown) in order to produce 
appropriate final reports regarding structure and defects of an object inspected. 

The CLRP unit 120 preferably comprises the following subunits: 

1. A materia] identification subunit 170 which preferably receives the 
color image data input to the CLRP unit 120 and is preferably operative to derive 
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therefrom information about materials comprised in an object of which the input image 
is the image. A preferred implementation of the material identification subunit 170 is 
described in more detail below with reference to Fig. 2. 

2. A material contours subunit 180 which preferably receives an output 
from the material identification subunit 170 and is preferably operative to derive 
therefrom contour information representing contours and/or borders in the input image. 
A preferred implementation of the material contours subunit 180 is also described in 
more detail below with reference to Fig. 2. 

It is appreciated that, in a case where the input image does not comprise 
an image of an object that the term "material" as used throughout the present 
specification and claims, particularly but not exclusively with respect to the material 
identification subunit 170 and the material contours subunit 180, is used to refer to 
features in the input image which, upon analysis, appear to correspond to features of one 
or more materials in an object which the image would represent, if the image did 
represent an object. In other words, with a broadened understanding and definition of 
the term "material", it is believed that the operations performed by the material 
identification subunit 170 and the material contours subunit 180 are also useful in 
analyzing an image which does not comprise an image of an object. 

3. A color surface defects subunit 190 which preferably receives the color 
image data input to the CLRP unit 120 and is preferably operative to derive, from the 
input, information about surface defects in the input image, the surface defects typically 
corresponding to defects in an object of which the input image is an image. A preferred 
method of operation of the color surface defects subunit 190 is described in more detail 
below with reference to Fig. 12. 

The CABS unit 130 preferably comprises the following subunits: 
1. A binarization and eel production subunit 200 which preferably 
receives the monochrome image data input to the CABS unit 130 and is preferably 
operative to derive therefrom binary information representing a binary image 
corresponding to the input image, based on the monochrome image data. Additionally, 
in a preferred embodiment of the present invention the binarization and eel production 
subunit 200 receives a reduced color image from the material identification subunit 170, 
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and is operative to derive therefrom binary information representing a binary image 
corresponding to the input reduced color image. The binarization and eel production 
subunit 200 also is preferably operative to define borders within the image, preferably 
by means of contour elements, also known as eels, and to output information comprising 
5 a description of the eels produced. A preferred method of operation of the binarization 
and eel production subunit 200 is similar to that described below for one component of a 
color image, with reference to Fig. 7. 

The term "binary" in all of its forms, as used throughout the present 
specification and claims and particularly in describing information, an image, or a 

1 0 method for processing information or processing an image, refers to having two states or 
two outcomes. In the case of an image, for example, a binary image is an image in 
which each element, such as a pixel, comprised in the image has only two possible 
states, which may be represented as "black" and "white" or, alternatively, "0" or "1". 
The term binary, in referring to images or image information, is thus used in contrast to 

1 5 the terms "gray scale" and "color", which are well-known in the art and which refer to 
particular types of images in which there are more than two possible states or outcomes 
for each element. 

2. A nicks and protrusions unit 220 which preferably receives the eel 
information output by the binarization and eel production unit 200 and is preferably 

20 operative to derive therefrom information about small border irregularities, known as 
nicks and protrusions, in borders defined by the binarization and eel production unit 
200, and to report the nicks and protrusions as output. A preferred method of operation 
of the nicks and protrusions subunit 220 is described in more detail below with 
reference to Figs. 33A and 33B. 

25 3. A gray small defects subunit 210 which preferably receives the 

monochrome image data input to the CABS unit 130 and is preferably operative, using 
the monochrome image data, to identify small defects in the input image which may, in 
a case where the image is an image of an object, indicate defects in the object of which 
the image is an image. Methods of identifying small defects using monochrome 

30 information are well known in the art and are described, for example, in US Patent 
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5,586,058 to Aloni et al., referred to above, the disclosure of which is hereby 
incorporated herein by reference. 

The SKFD unit 140 preferably comprises the following subunits: 

1. A line and space skeleton subunit 230 which preferably receives the 
5 binary information output from CABS unit 130, typically via the binarization and eel 

production subunit 200, and is preferably operative to derive therefrom and to produce 
as output line and space skeleton information describing the input image. Typically, the 
line and skeleton information is derived using a pixel consumption method, as is well 
known in the art. A preferred method of operation of the line and space skeleton subunit 
1 0 230 is described in more detail below with reference to Fig. 27. 

2. A morphology subunit 240 which preferably receives the line and 
space skeleton information output by the line and space skeleton subunit 230 and is 
preferably operative to identify morphological features of the input image based on the 
line and space skeleton information and to produce information describing the 

1 5 morphological features as output. A preferred method of operation of the morphology 
subunit 240 is described in more detail below with reference to Figs. 30A - 30C. 

The following definitions and general discussion may be of assistance in 
understanding the present specification and claims. 

A population of pixels which have largely homogeneous color 
20 characteristics is termed herein a "largely homogeneous color population". Typically, a 
color image to be analyzed comprises, as is well-known in the art, a plurality of regions 
of largely homogeneous color population. 

It is well known in the art to represent a color image in a color format or 
color space, such as, for example: RGB format, comprising separate color values for 
25 red, green, and blue; and HSI format, comprising separate values for hue, intensity, and 
saturation. For the sake of simplicity of description, throughout the present specification 
and claims, unless stated otherwise, the example of RGB format will be used, it being 
appreciated that other formats may, in most cases, also be used. 

It is well known in the art that a single color component of an RGB 
30 image may considered separately from other components of the RGB image; for 
example, the green component may be considered separately. A derived image 
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comprising only the green component may be considered, for the sake of simplicity of 
processing and description, to be a monochrome image. In discussing such a single 
component derived image, the value of a pixel may be referred to as a gray scale value 
for the sake of simplicity of description, even though the value is in some sense a "green 
scale value". 

In the field of grayscale image analysis, the concept of gradient is well 
known and corresponds generally to a first derivative of a function of pixel values, 
indicating the rate of change of pixel value at a predetermined location. It will therefore 
be evident to persons reasonably skilled in the art that areas of homogeneous color in a 
single color component, such as the red component, will generally comprise pixels 
having a zero or near-zero gradient value. Areas of non-homogeneous color, including 
borders and edges, will generally comprise pixels having a significantly non-zero 
gradient value. 

Preferred prior art methods which may be used in determining an edge in 
a grayscale image, including determination to sub-pixel accuracy in a grayscale image, 
include second derivative computation methods, which typically use a difference of 
Gaussians (DOG) method to calculate a second derivative approximation of a pixel 
intensity function. Typical prior art methods are described in the following US Patents, 
the disclosures of which are hereby incorporated herein by reference: 

1. US Patent 5,774,572 to Caspi, which describes an automatic visual 
inspection system, which is preferably operative to convolve a 2-dimensional digital 
gray scale image of an object with a filter function related to the second derivative of a 
Gaussian function forming a 2-dimensional convoluted image have signed values. The 
location of an edge in the object is achieved by finding zero crossings between adjacent 
pixels having oppositely signed intensity values. 

2. US Patent 5,774,573 to Caspi et al, which describes a visual inspection 
system which uses convolution of a 2-dimensional digital gray scale image of an object 
with a filter function related to the second derivative of a Gaussian function forming a 
2-dimensional convolved image having signed pixel intensity values. The convolution 
of Caspi et al. can be performed with a difference-of-two-Gaussians, one positive and 
one negative. 
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Color Image Processing 
Reference is now made to Fig. 2, which is a simplified block diagram 
illustration of a preferred implementation of the material identification subunit 170 and 
the material contours subunit 180 comprised in the CLRP unit 120 of Fig. 1. 
5 Generally, the task of the material identification subunit 170 is to 

produce, from color information of an input image, as much information as possible 
about homogeneous color population regions in the input image. In the material 
identification subunit 170, a region of homogeneous color population is generally 
identified with a particular material or combination of materials as may result from the 

10 presence of a translucent or partially transparent coating or overlay covering another 
material. For example, when the object being inspected is a BGA, typical optically 
distinguishable materials and materials combinations include copper conductor, copper 
conductors that are coated or plated with gold, substrate, and those parts of any of the 
aforementioned materials which are covered by a solder-mask overlay. It is appreciated 

15 that other objects being inspected might include other materials and material 
combinations such as, for example, silver. Therefore, the primary task of the material 
identification subunit 170 is to identify, in the input image and based on the color 
information, areas of homogenous color population. 

The term "translucent overlay", as used throughout the present 

20 specification and claims, includes, for example, a translucent or partially transparent 
solder mask or, for example, a translucent substrate through which a conductor located 
on an opposite side thereof may be seen. 

Generally, areas of homogenous color population are referred to herein as 
representing areas of a particular material, which is generally the case when an input 

25 image to be analyzed is an image of a physical object. It is appreciated, however, as 
explained above, that the present invention is not limited to analyzing only input images 
which are images of a physical object, and that the use of the term "material" in a case 
where an image may not be an image of a material object is by way of analogy and is not 
meant to be limiting. 

30 Generally, the task of the material contours unit 180 is to produce, from 

color information of an input image and from the output of the material identification 
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subunit 1 70, an indication of borders between areas of homogenous color populations, 
said borders preferably being identified at a subpixel level. 

The apparatus of Fig. 2 preferably comprises the following: 
1 . a material classification unit 171; 
5 2. a color edge detector 1 72; 

3. a region expanding unit 173; 

4. a eel construction unit 174; and 

5. a contourization unit 175. 

The material classification unit 171 preferably receives color component 

10 data, typically comprising RGB values for each pixel in the input image, and is 
preferably operative to classify areas of largely homogenous color population as 
belonging to a particular material. In a preferred implementation, the material 
classification unit 171 is implemented as a hardware LUT, so that classification of each 
pixel is based directly on the color component values, typically RGB values, of that 

15 pixel, so that RGB values falling within a color space associated with a particular 
material or material combination will be associated with that material. 

Preferably, in a typical case where a plurality of images of objects 
composed of similar materials is to be analyzed, the LUT may be built by analyzing a 
representative image using methods well known in the art. It is appreciated that a 

20 number of appropriate methods of analyzing a representative image are well known in 
the art, including simple methods based on histogram-like pixel color distribution 
analysis, and that any appropriate method may be used. A preferred method for 
performing such an analysis is described in the following reference, the disclosure of 
which has been incorporated herein by reference: Dorin Comaniciu and Peter Meer, 

25 "Distribution Free Decomposition of Multivariate Date", SPR '98 Invited Submission, 
Department of Electrical and Computer Engineering, Rutgers University, Piscataway, 
NJ 08855, USA. 

The materials classification unit 171 preferably outputs classified 
materials information indicated an assignment of each pixel in the input image to a 
30 particular material. 
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The color edge detector 172 preferably receives the same input as the 
materials classification unit 171. The color edge detector 172 is preferably operative to 
compute an approximate first derivative, typically by applying a Sobel operator, as is 
well known in the art, typically to HSI components of the input image. The Sobel 
operator is preferably applied separately to each of the HSI components, and then a 
combined result is derived as explained below. 

Sobel edge detection is well known in the art. The magnitude of the 
gradient of a function q> is given by the sum of the magnitudes of the partial derivatives 
of cp in the x and y directions, which is approximated in Sobel estimation at a point (ij) 
as follows: 

I (p(i-l j-l) + 2cp(ij-l) + <p(i+i j-l) - (cp(i-l + 2cp(ij+l) + cp(i+l j+1)) | + 
i cp(i-l j-l) + 2<p(i-l,j) + cp(i-l J+1) - (<p(i+l j-l) + 2<p(i+l j) + q>(i+l j+l)) | 

Sobel edge detection in the present invention preferably uses a circular 3 
x 3 edge detector and, via image convolution, calculates a 2D gradient magnitude as a 
sum of absolute values of vertical and horizontal gradient components, in accordance 
with the above formula. Typically, the following kernels may be used for image 
convolution: 

1 . Vertical gradient kernel: 

-1 0 1 
-2 0 2 
-1 0 1 

2. Horizontal gradient kernel: 

1 2 1 
0 0 0 
-1 -2 -1 

The color edge detector 1 72 then preferably computes a color edge as a 
weighted sum of H, S, and I component gradients as follows: 
|G C olor| = (|Gh| Relevance(I)WeightH + 

|G s |Relevance(I) Weights + |G,|WeghfI) 

where: 

Gcolor is the color edge; 
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Gh is the gradient of hue; 

Gs is the gradient of saturation; 

G| is the gradient of intensity; 

Relevance(I) is a color relevance weight function of intensity; 
5 WeightH is a weight for the hue component; 

Weights is a weight for the saturation component; and 
Weightl is a weight for the intensity component. 

Further discussion of the factors mentioned immediately above is found 
in Philippe Pujas and Marie- Jose Aldon, "Robust Color Image Segmentation", referred 

1 0 to above, the disclosure of which is hereby incorporated herein by reference. 

Reference is now made to Fig. 3, which is a simplified flow chart 
illustration of a preferred method of operation of the color edge detector 172. The flow 
chart of Fig. 3 is self-explanatory with reference to the above discussion. 

Referring back to Fig. 2, the region expanding unit 173 preferably 

15 receives the outputs produced by the material classification unit 171, comprising 
classified materials information, and the output produced by the color edge detector 172, 
comprising color gradient information. The region expanding unit 173 is preferably 
operative to extend each region comprising a homogeneous color population as 
identified by the material classification unit as far as possible, so that the borders 

20 between homogeneous color populations are as narrow as possible. 

Reference is now additionally made to Fig. 4, which is a simplified 
flowchart illustration of a preferred method of operation of the region expanding unit 
173 of Fig. 2. In the method of Fig. 4 unassigned pixels are assigned to a neighboring 
region of homogeneous color which has a low gradient, indicating that the unassigned 

25 pixel is, in so far as possible, assigned to a neighboring region having the least clear 
border with the unassigned pixel. The method of Fig. 4 preferably includes the 
following steps: 

A gradient is computed at each pixel in the image using an appropriate 
edge extraction operator as described above (step 330). A current gradient level, 
30 indicating a gradient level of unclassified pixels to be classified at the current stage of 
the method of Fig. 4, is set to 0 (step 340). 
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The following steps are preferably performed for each unclassified pixel 
whose gradient is less than or equal to the current gradient level (step 350) following 
stepwise incrementing. 

Among neighboring pixels of the unclassified pixel, the neighboring 
pixel having the smallest gradient is identified (step 360). The neighboring pixels may 
comprise 4 pixels comprising 2 horizontal neighboring pixels and 2 vertical neighboring 
pixels. Alternatively, the neighboring pixels may also comprise 4 diagonal neighboring 
pixels, thus comprising 8 pixels. 

The unclassified pixel is assigned to the pixel among the neighboring 
pixels which was identified in step 360 as having the smallest gradient (step 370). If 
there is more than one neighboring pixel having the smallest gradient, the first appearing 
of the more than one neighboring pixels having the smallest gradient is chosen and used. 

A check is made as to whether all pixels have now been classified (step 
380); if so, the method of Fig. 4 preferably concludes. 

If at least one unclassified pixel remains unclassified, the current gradient 
level is incremented (step 390), and execution of the method of Fig. 4 continues with 
step 350. 

It is appreciated, as stated above, that most components of the system of 
Fig. I, including the apparatus of Fig. 2, are preferably implemented in hardware for 
optimum performance. It is appreciated that a skilled person of the art would implement 
the method of Fig. 4 in hardware using well-known techniques appropriate to optimized 
hardware implementation of software algorithms. 

The eel construction unit 174 preferably receives the color component 
input, typically comprising RGB input, received by the materials classification unit 171 
and the color edge detector 172. The eel construction unit 174 is preferably operative to 
distinguish borders among regions of different homogeneous color populations and to 
determine therefrom eels representing such borders determined at a sub-pixel level 
separately in each of the R, G, and B gray scale images. Cels, also known as contour 
elements, are described in more detail below. 

Appropriate apparatus and methods for making such a sub-pixel 
determination in a gray scale image are described in US Patent 5,774,572 to Caspi and 
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US Patent 5,774,573 to Caspi et al, also referred to herein as "the Caspi patents", both 
of which have been referred to above and the descriptions of which have been 
incorporated herein by reference. It is appreciated, in the present invention, that 
determination of a location of a border at a sub pixel level in a single-component gray 
5 scale image, such as a red component gray scale image, may be performed using the 
methods described in the Caspi patents. 

As is well known in the art, particularly in the Caspi patents referred to 
above, convolution with an appropriate difference of Gaussians (DOG) operator may be 
used to compute a second derivative and thus determine border location; in particular, a 

10 large and a small operator may be used. Preferred kernels include 3x1,5x1, and 5x3 
kernels. Reference is now additionally made to Figs. 5 and 6, which show examples of a 
preferred 5x5 large kernel and a preferred 3x3 corresponding small kernel, 
respectively. Figs. 5 and 6 are self-explanatory. 

The contourization unit 1 75 preferably receives the R, G, and B eels from 

1 5 the eel construction unit 1 74 and the segmented materials information from the region 
expanding unit 173 and is preferably operative to produce therefrom color eels 
representing borders between adjacent materials in the input image. 

Reference is now additionally made to Fig. 7, which is a simplified block 
diagram illustration of a preferred implementation of the contourization unit 175 of Fig. 

20 2. The apparatus of Fig. 7 preferably comprises the following: 

1. a material eel builder 176 which preferably receives the segmented 
materials information and is preferably operative to produce therefrom material eels 
which represent borders with pixel accuracy; 

2. a selector decision envelope builder 177 that builds an envelope, such 
25 as a 1 pixel envelope, around each material eel, indicating regions within which a border 

is to be determined; and 

3. a best contour selector 178, preferably receiving the material eels from 
the material eel builder 176 and the envelope from the selector decision envelope 
builder 177, as well as the R, G, and B eel information, and preferably operative to 

30 produce therefrom a final color eel representing the best border between adjacent 
materials. 



49 



.i.G n 3 ,i, o - 'z ... a jl .1. ■□: : ; ? 

WO 01/07893 PCT/TLOO/00434 



In determining a border in the best contour selector 178 it is appreciated 
that it is generally difficult or impossible to determine a border with sub pixel accuracy 
in the vicinity of a junction of edges and therefore it is preferred to use the 
contourization border in the vicinity of a junction. The vicinity of a junction may be 
5 defined in any appropriate manner such as, for example, as within a 3 pixel square 
neighborhood of a junction. 

In a preferred embodiment of the current invention, a decision as to 
whether to use a sub-pixel R, G, or B border or whether to use a pixel material eel 
border in areas that are not near a junction is made according to the materials which fall 

1 0 on both sides of the border. Certain materials are known to be best distinguishable, for 
example, in red; if those materials fall on the two sides of the border, then the red 
sub-pixel border is used to fix the border at that location. Other materials may not be 
distinguishable in any of R, G, or B, in which case a pseudo color may be generated, for 
example by weighting the RGB inputs in a predetermined manner, for border 

1 5 determination, or a material eel border would be used at that point. 

Reference is now made to Fig. 8, which is a simplified flowchart 
illustration of a preferred method of operation of the best contour selector 178 of Fig. 7. 
The method of Fig. 8 is self-explanatory with reference to the above discussion. 

Preferably, sup-pixel accuracy borders as described above are represented 

20 as a plurality of contour elements or eels, each eel indicating a sub-pixel accuracy border 
segment in the vicinity of a single pixel; each eel is roughly equivalent to a small vector 
of length approximately equal to one dimension of a pixel. One preferred method of 
defining a eel, including defining the location and orientation of the eel and the 
homogeneous color population on each side of the eel, wherein each population is 

25 correlated to a material or material combination as described above, is as follows: 
description typical number of bits for storage 

material edge index into a table indicating which materials are on each 

side of the eel 

direction 0 if the area of negative DOG is to the left of the eel when 

traveling from a first end of the eel to a last end of the eel, 
1 otherwise 
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edge code indication of direction of the eel, encoding the first edge 

and last edge of the pixel 
first edge intersection point, in a cyclic coordinate system, of the eel 

with the first edge 

last edge intersection point, in a cyclic coordinate system, of the eel 

with the last edge 

Reference is now additionally made to Figs. 9-11, which are simplified 
pictorial illustrations of preferred eel construction and nomenclature, useful in 
understanding the present invention. The illustration of Fig. 9 comprises a plurality of 
pixels 530 and a second plurality of virtual eel pixels 540. Typically, each virtual eel 
5 pixel 540 is of the same size as each pixel 530 and each virtual eel pixel 540 is centered 
about a corner of four pixels 530. In Fig. 9, by way of example only, each of four pixels 
530, which comprise neighboring pixels of a central virtual eel pixel 545, are shown as 
having given gradient values of -100, +20, +50, and +20 respectively. 

The illustration of Fig. 9 also comprises a eel 550. The location of a first 

10 edge intersection point 560 of the eel 550 is preferably fixed by linear interpolation 
between the adjacent eel gradient values, for example +20 and -100 as shown, in order 
to estimate the zero-crossing point, as is well-known in the art. Similarly, the location of 
a last edge intersection point 570 of the eel 550 is preferably fixed by linear 
interpolation between adjacent eel gradient values, for example +50 and -100 as shown. 

15 Fig. 10 depicts a preferred circular coordinate system for describing the 

first edge and last edge of the eel 550, as well as an edge numbering system in which the 
top, left, right, and bottom edges of the cell 550 are numbered 0, 1, 2, and 3 
respectively. Fig. 1 1 depicts a preferred scheme for describing the edge code indicating 
the direction of the eel 550 of Figs. 9 and 10. 

20 A preferred method of determining edge code and direction based DOG 

sign values in four neighboring pixels would be to perform a lookup table based on the 
following: 

DOG sign edge code Direction 
+ +7 0 
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In the immediately preceding list of edge codes, it is appreciated that 
code 6 represents a saddle and code 7 represents "no eel". 

The first and last edges of a eel, according to the numbering system of 
Fig. 10, may be determined from the following: 
edge code number of number of 



first edge last edge 

0 0 1 

1 0 2 
0 0 3 

3 I 2 

4 1 3 

5 2 3 

6 2 3 
undefined undefined 



In a preferred embodiment of the present invention, the eels, which are 
calculated as described above, are collected into an image map representing the eel 
borders between homogeneous color populations, and indicating for each eel which of 
the colors of the homogeneous color population is located on either side of the cel. It is 
readily appreciated that by correlating materials to homogeneous color populations as 
hereinabove described, color eel image maps include data as to the identity and location 
of materials present on the object being inspected. 

In a preferred embodiment of the present invention, the collection of eels 
is preferably included in a report of features of the object inspected, which may be 
further processed as desired. An additional report preferably includes a reduced color 
map, specifying regions of homogeneous color population, of the object being inspected, 
which may be used for example by the binarization and eel production unit 200 to 
produce binary eels in order to generate a color morphology map, as well be described 
in greater detail below. Additionally, color eels may be analyzed to ascertain the shape 
and location of various coatings, such as partially transparent overlays, and output defect 
reports when detected as being defective. 



53 



..ii. kJ; ','l';> .,}'... '•',..;( ' ; - ":■ ':;.'!■ .„ U' ,„!:., Ji, ' !!,..'■■ 
WO 01/07893 PCT/IL00/00434 



Surface Defect Detection 
Reference is now made to Fig. 12, which is a simplified block diagram 
illustration of a preferred implementation of the color surface defects subunit 190 of Fig. 
1 . For brevity of reference, the color surface defects subunit 1 90 is sometimes referred 
5 to herein as COMRADD 190. 

Surface defects, also known as area defects can be described as local 
abnormalities within a randomly textured surface, and typically include: residues such as 
process residues remaining on the surface of an article being inspected, voids in various 
coatings and platings, irregularities in the surface topography of the article including and 
10 in particular in metal coatings, scratches, foreign materials, stains, and oxidation of 
various metals and metal coatings. Surface defects are preferably classified into two 
groups: 

- high contrast defects, which generally comprise defects with well 
defined edges on which a strong gradient can be measured, such as a scratch or a pit; 

15 and 

- low contrast wide area defects, which generally comprise defects with 
unclear edges, on which a relatively weak gradient is measurable, such as a water stain 
or an oxidized region. 

Defects may also be classified in other ways, such as: 
20 - chromatic defects, which generally differ from their surroundings 

mainly by color; oxidized regions are an example of chromatic defects; and 

- achromatic defects, or intensity oriented defects, that is, defects which 
are either brighter or darker than their surrounding, such as, for example, pits, scratches, 
and stains. 

25 Generally, the apparatus of Fig. 12 seeks to detect surface defects as 

described above, even on topologically complex surfaces comprising a variety of 
materials and textures. Preferably, the apparatus of Fig. 12 is designed to operate inside 
a material, ignoring the edges thereof, and thus is designed to receive an input 
segmented into materials and edges, such as the output produced by the material 

30 contours subunit 180, described above. Preferably, the apparatus of Fig. 12 achieves its 
results using multi-resolution detecting, applying chromatic and achromatic detectors in 
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parallel at different resolutions. The apparatus of Fig. 12 can preferably be configured to 
operate at different sensitivities for different materials, and at different sensitivities as a 
function of distance from material edges. Preferably, the apparatus of Fig. 12 produces a 
report of detected surface defects. Preferably, the surface defects are considered 
5 candidate defects, and the report is accompanied by a snapshot image of each possibly 
defective location in the image being inspected; the report and the snapshot image may 
be further processed as desired. 

Thus, it will be appreciated that, in the present invention, preferred inputs 
to the apparatus of Fig. 12 include the color image data defining the object or image 

10 being inspected, material information as produced by the material identification subunit 
170, and color eel information as produced by the material contours unit 180. 

Preferably, for purposes of efficiency of operation, the apparatus of Fig. 
12 is preferably implemented partly or wholly in special purpose hardware, as is 
well-known in the art, with the remainder, if any, being implemented in software and 

1 5 general purpose hardware. 

The apparatus of Fig. 12 preferably comprises the following units: 
1. A colorl/color2 creation unit 400 which preferably receives the color 
information and the material information and is preferably operative to derive therefrom 
two derived color images, known herein as colorl and color2. 

20 Preferably in the present invention, a preferred color is predefined for 

detecting chromatic defects in each material, and a preferred color is predefined for 
detecting achromatic defects in each material. The colorl image is then defined as the 
best image for detecting achromatic defects, and the color2 image is similarly defined as 
the best image for detecting chromatic defects. Each of the colorl image and the color2 

25 image preferably comprises a collage in which each pixel, according to its associated 
material, is assigned a pixel taken from one of four possible gray scale images: R, G, B, 
and one of the HSI transformations thereof; the chosen one of the three HSI 
transformations is preferably chosen to be effective for the materials which are expected 
to be present in the image (Fig. 13). 

30 Reference is now additionally made to Fig. 13, which is a simplified 

flowchart illustration of a preferred method of operation of the colorl/co!or2 creation 
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unit 400 of Fig. 12. The method of Fig. 13 is self-explanatory with reference to the 
above discussion. 

2. A directional enable images creator 402 which receives the color eel 
information and is preferably operative to produce two directional enable images, okdir 
5 and sub-okdir, which are preferably used to control further operation of the apparatus of 
Fig. 12. It will be appreciated by persons skilled in the art that, in applying a sensitive 
defect detector on a poly-material image it is very desirable to avoid false alarms that 
originate from material edges. 

Strong gradients are generally measured on material edges; therefore, in 

10 the present invention, particularly with respect to surface defect detection, it is desirable 
to disable gradient measurement at or near material edges. The following strategy is 
used in the present invention to make it possible to find defects near material edges 
without incurring false alarms from gradients associated with the material edges. The 
image is preferably divided into three main groups: on the edges, no measurements are 

1 5 taken; near the edges, measurements are only taken in a direction parallel to the local 
edge direction, this direction of measurement being termed herein the "okdir" direction; 
in other places, normal measurements are performed. Preferably, an okdir map is used 
for full resolution measurement, while a derived "sub okdir" map is used for sub 
resolution measurement. 

20 The term "sub resolution", as used herein, refers generally to the use of a 

sub sampled image, such as an image in which only every nth pixel in each dimension, 
such as every 5th pixel in each dimension, of the original image is used. By "shrinking" 
the image using a sub sampled image, it is appreciated, as is well known in 
mathematics, that certain computed values such as gradient values are increased, since 

25 distances shrink without pixel value shrinking. In this way, low contrast defects may 
become more apparent. 

A preferred implementation of the portion of the directional enable 
images creator 402 which creates okdir is described in more detail below with reference 
to Fig. 14, and of the portion which creates sub okdir below with reference to Fig. 21. 

30 3. and 4. Multi-resolution gradient measuring apparatus comprising a sub 

resolution channel low contrast defect detector 404 and a full resolution channel high 
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contrast defect detector 406, each preferably receiving the colorl and color2 images, the 
low contrast detect detector 404 preferably receiving the sub-okdir enable image and the 
high contrast detect detector 406 preferably receiving the okdir enable image, each of 
the low contrast detect detector 404 and the high contrast defect detector 406 being 
5 operative to perform gradient analysis and to produce a report of defects. Preferably, a 
total of four edge detectors are employed, one each for each of the colorl and color2 
images in each of the low contrast defect detector 404 and the high contrast defect 
detector 406. In addition, two small defect detectors are preferably employed only in the 
full resolution channel high contrast defect detector 406, one for each of the colorl and 

10 color2 images. Thus, preferably a total of six detectors are employed. 

5. A channel reports merger 408 which is preferably operative to receive 
reports from each of the low contrast defect detector 404 and the high contrast defect 
detector 406 and to produce therefrom an appropriate merged report. 

Reference is now made to Fig. 14, which is a simplified block diagram 

15 illustration of a preferred implementation of a first portion of the directional enable 
images creator 402 of Fig. 12. The portion of the directional enable images creator 402 
illustrated in Fig. 14 is the portion which creates okdir; the portion which creates sub 
okdir is described below with reference to Fig. 21 . 

The apparatus of Fig. 14 preferably comprises an edge enable image 

20 creator 410, which preferably receives color eel information as input and is preferably 
operative to derive therefrom an edge enable image. In the edge enable image all pixels 
which "touch" any eel are preferably marked as edge pixels while all other pixels are left 
unmarked, representing material pixels; thus the edge enable image is created as a 
binary image. 

25 Reference is now additionally made to Fig. 15, which is a simplified 

pictorial illustration of an image pixel and neighboring virtual eel pixels, useful in 
understanding the operation of the apparatus of Fig. 14. Typically, for each image pixel 
such as an image pixel 690, a neighborhood of four virtual eel pixels centering around 
the image pixel 690, such as virtual eel pixels 700, 710, 720, and 730 is considered. The 

30 virtual eel pixels 700, 710, 720, and 730 are typically numbered 0, 1, 2, and 3 
respectively according to their position. 
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If no eel is found in any of the four neighboring virtual eel pixels 700, 
710, 720, and 730 the image pixel 690 is not considered to be an edge pixel. If a eel is 
found in any of the four neighboring virtual eel pixels 700, 710, 720, and 730, the image 
pixel 690 is considered to be an edge pixel unless all such eels are excluded eels 740 as 
indicated in Fig. 15. In accordance with Fig. 10 and the description thereof above, the 
excluded eels 740 comprise: 

virtual eel picture number reference number excluded eel number 

0 700 0 

1 710 1 

2 720 4 

3 730 5 

Referring back to Fig. 14, the apparatus of Fig. 14 preferably comprises a 
morphology control unit 412, typically implemented as a LUT, which is preferably 
operative to provide control information to a plurality of erosion units 414, typically 
comprising 3 erosion units 414. Each of the 3 erosion units 414 is preferably operative 
to apply erosion, as is well known in the art, to the edge enable image created by the 
edge enable image creator 410. Typically, a first erosion unit 414 applies 2 levels of 
erosion and produces an enable2 map; a second erosion unit 414 applies 4 more levels 
of erosion and produces an enable6 map; and a third erosion unit 414 applies 4 more 
levels of erosion and produces an enable 10 map. Typically, except for their inputs and 
control information received, each of the three erosion units 414 is identical. 

Preferably, each erosion step is carried out according to an erosion key 
having a value; one possible example of an erosion key value is as follows: "0C CCCT 
CC00". Preferably, each character of the erosion key represents a single erosion step, 
reading from left to right, each step being preferably interpreted according to an erosion 
operator as follows: 



key character description of erosion operator 

0 do not erode, output=input 

S save, do not consume open ends 

C do not break connectivity 

N allow breaking of connectivity 
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T smooth edge by eating small protrusions 

Preferably, all of the erosion operators comprise orthogonal operators, 
that is, operators that consume a central pixel if one of the central pixel's four first order 
neighbors is already eaten. It is appreciated that the enable2 map is preferably produced 
by operations "0C", the enable6 map by further operations "CCCT", and the enable 10 
map by further operations "CC00". It will be appreciated by persons skilled in the art 
that the "0" erosion steps are preferably included for ease of certain hardware 
implementations. 

The enable maps are preferably used to extend the range, within an edge, 
from which the edge direction can be determined. The erosion operations tend to 
increase the size of the original edge enable image, thus allowing a determination of 
edge detection at a greater distance from the edge; preferably, an okdir-envelope of up to 
10 pixels width may be created. 

The apparatus of Fig. 14 also preferably includes an okdir map 
computation unit 416, which is operative to produce an okdir image. Reference is now 
additionally made to Fig. 16, which is a simplified flowchart illustration of a preferred 
method of operation of the okdir map computation unit 416. The method of Fig. 16 
preferably comprises the following steps: 

A relevant enable image, from among the 3 enable images, is determined 

(step 4 18). 

Reference is now additionally made to Fig. 17, which is a simplified 
flowchart illustration of a preferred implementation of step 418 of Fig. 16. The 
following definitions may be useful in understanding the method of Fig. 17: 

- envelop pixels: pixels that reside on the borders of a 5 x 5 matrix; 

- okdir address: envelop and map code, as described below; 

- envelop: a bit map of the envelope pixels of a chosen enable image, the 
chosen enable image being chosen from among the enable2 image, the enable6 image, 
and the enable 10 image, preferably in accordance with the method of Fig. 17; 

- map code, a code, typically implemented as a 2 bit code, describing a 
chosen enable map, typically according to the following interpretation: 

code interpretation 
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00 



central pixel is an edge pixel 



01 



envelop taken from enable2 image 
envelop taken from enable6 image 
envelop taken from enable 10 image 



02 



03 



The method of Fig. 1 7 preferably receives as input the 3 enable images 
comprising the enable2 image, the enable6 image, and the enablelO image and 
preferably comprises the following steps: 

A check is made as to whether the central pixel of the enable2 image is 
5 an edge pixel (step 420). If so, okdir address is set to 0 (step 422); that is, both envelop 
and map code are set to 0, and the method of Fig. 17 terminates. 

Otherwise, a check is made as whether the envelop pixels of the enable2 
image include at least one edge pixel (step 424). If so, envelop is set equal to the 
enable2 envelop, map code is set equal to 01 (step 426), and the method of Fig. 17 
10 terminates. 

Otherwise, a check is made as to whether the envelop pixels of the 
enable6 image include at least one edge pixel (step 428). If so, envelop is set equal to 
the enable6 envelop, map code is set equal to 02 (step 430), and the method of Fig. 17 
terminates. 

15 Otherwise, envelop is set equal to the enablelO envelop and map code is 

set equal to 03 (step 432). 

Generally, it will be appreciated by persons skilled in the art that the 
method of Fig. 1 7 is operative to choose the envelop of the smallest enable map which 
includes an edge pixel, unless the central pixel of the enable2 image is an edge pixel, in 
20 which case no envelop is chosen. 

Referring back to Fig. 16, the okdir is computed (step 419). Preferably, 
for reasons of efficiency as described above, step 419 is implemented in hardware in a 
LUT. Reference is now additionally made to Fig. 18, which is a simplified flowchart 
illustration of a preferred implementation of step 419 of Fig. 16. The method of Fig. 18 
25 preferably includes the following steps: 

A check is made as to whether okdir address equals 0 (step 434); if so, 
okdir is also set equal to 0 (step 436). 
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A check is made as to whether all of the envelop pixels are occupied 
(step 438); if so, okdir is set equal to 15, typically represented as a hexadecimal number 
OxF (step 440). 

The enable image is then eroded to skeleton heads (step 442); a skeleton 
5 head is a single isolated pixel which remains after erosion. Reference is now 
additionally made to Fig. 19, which is a simplified pictorial illustration of an example of 
a process of erosion resulting in one skeleton head 452. 

Referring back to Fig. 18, an action is taken based on one of three 
possible cases concerning the number of skeleton heads (step 444): 
10 If there is 1 skeleton head, okdir is perpendicular to a vector 

connecting the skeleton head and the central pixel of the spread (step 446). This case of 
okdir direction is shown as reference number 454 in Fig. 19. 

If there are 2 skeleton heads, okdir is the direction of a line drawn 
between the two skeleton heads (step 448). 
1 5 If there are 3 or more skeleton heads, okdir is set to 0, indicating 

that no measurement is to be taken. 

Reference is now additionally made to Fig. 20, which is a table showing 
okdir values, corresponding directions of measurement, and a preferred gradient 
operator for measuring. In the table of Fig. 20, "LinearDiff ' refers to a linear differential 
20 operator, as is well known in the art, while "HalfSobel" refers to a modified Sobel 
operator, called a half Sobel operator, which is described in more detail below. The 
table of Fig. 20 is otherwise self-explanatory. 

Reference is now made to Fig. 21, which is a simplified flowchart 
illustration of a preferred method of operation of a second portion of the directional 
25 enable images creator 402 of Fig. 12; the method of Fig. 21 is directed towards 
production of the sub okdir image. 

The method of Fig. 21 preferably comprises the following steps: 
An enable 10 image, produced as described above with reference to Fig. 
14, is received as input and is expanded (step 456). Preferably, expanding the enable 10 
30 image includes performing a 5 x 5 spread and, if any one of the pixels in the expansion 
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is an edge pixel, setting the central pixel of the spread to be an edge pixel. The image 

thus produced is referred to herein as a "baseenable" image. 

A k x k spread of the baseenable image is performed, with k preferably 

equal to 11 or 21 (step 458). A virtual 3x3 spread is then created from 9 predetermined 
5 pixels in the k x k spread (step 460). Reference. is now additionally made to Fig. 22, 

which illustrates a preferred choice of predetermined pixels for an 11 x 11 spread, it 

being appreciated that a similar choice of predetermined pixels may be made for a 21 x 

21 spread or for another spread. It will further be appreciated that the choice of 9 pixels 

from the k x k comprises a form of subsampling. 
10 A set of heuristics is applied in order to determine the sub okdir value 

from the virtual 3x3 spread (step 462). Reference is now additionally made to Fig. 

23 A, which is a table illustrating preferred heuristics for use in step 462 of Fig. 21, and 

to Fig. 23B, which is a simplified pictorial illustration showing a naming convention for 

the 9 pixels of the 3x3 virtual spread, the names of Fig. 23B being used in Fig. 23 A. 
15 Figs. 23 A and 23B are self-explanatory. 

Reference is now made to Fig. 24, which is a simplified flowchart 

illustration of a preferred method of operation of a portion of the apparatus of Fig. 12. 

The flowchart of Fig. 24 comprises a preferred method of operation of the full resolution 

channel 406, the sub resolution channel 404, and the channel reports merger 408 of Fig. 
20 12. Portions of the flowchart of Fig. 24 relating to the aforementioned portions of Fig. 

12 are indicated generally in Fig. 24 by reference numbers 404, 406, and 408. 

Preferably, as described above with reference to Fig. 12, both the full 

resolution channel 406 and the sub resolution channel 404 receive as input both the 

colorl image and the color2 image. 
25 The method of Fig. 24 preferably comprises the following steps: 

1 . Steps relating to the full resolution channel 406: 

The input images are smoothed, preferably with a 3 x 3 mirror Gaussian 
filter (step 464). As is well known in the art, a Gaussian filter may be used to smooth an 
image and to reduce texture effects therein. It is appreciated that a Gaussian filter has a 
30 disadvantage in that pixels which are next to an edge may be influenced by neighboring 
edge pixels, effectively expanding the edge towards the material; this problem may be 
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referred to as an "edge effect". In the present invention, a mirror Gaussian operator, 
combining a Gaussian operator with a mirror operator which transforms pixels in a 3 x 3 
spread into the pixel on the opposite side of the 3 x 3 spread, is used to overcome the 
edge effect known in the prior art, and thereby to smooth image data relating to non 
border regions in a manner which is not influenced by image data relating to borders. 

Reference is now additionally made to Fig. 25, which is a simplified 
flowchart illustration of a preferred implementation of step 464 of Fig. 24, in which 
Gauss(ij) represents application a standard Gaussian function. The method of Fig. 25 is 
self-explanatory. 

Returning to Fig. 24, an appropriate gradient operator is applied (step 
466), the gradient operator typically comprising a half Sobel detector within material 
regions and a linear differential detector (LinearDiff) along edges. 

In the present invention applying a linear differential detector preferably 
comprises measuring in the okdir direction, or in the sub okdir direction in the case of 
the sub resolution channel, a difference in pixel value between the head "b" of a short 
vector and the tail "a" of the short vector. The value of the linear differential detector is 
preferably the value at "b" minus the value at "a". 

A Sobel operator, as is well known in the art and as described herein, is a 
gradient operator combining a gray level derivative along the principal horizontal and 
vertical axes, convoluted with a gray level derivation along the diagonals. As described 
below, in the method of Fig. 24 the Sobel operator is used in the sub resolution channel. 
Typically a Sobel operator may be implemented as two separately applied operators, a 
horizontal operator such as: 
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1 0 -1 

2 0-2 
1 0 -1 



and a vertical operator such as: 



1 


2 


1 


0 


0 


0 


-1 


-2 


-1 



In the present invention, an improved Sobel-type operator known as the 
half Sobel operator is used, preferably in step 466, with the purpose of adding to the 
properties of the Sobel operator detection of local minima and/or maxima. Typically, a 
half Sobel horizontal operator has the form: 



1 0 0 0 -1 
0 0 0 0 0 

2 0 7+4 0 -2 

0 0 0 0 0 

1 0 0 0 -1 



while a half Sobel vertical operator typically has the form: 
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Typically, a half Sobel operator is applied in two steps. Taking the 
following form of a half Sobel horizontal operator, where the entries refer to the values 
of the pixels falling under the operator, with the "-" symbol indicating that the operator 
has a zero value at that pixel location: 

5 

Al - Bl 

AO - C - BO 

A2 - - B2 

Let DifO - (2* AO + Al + A2) - 4 * C; let Difl = 4*C - (2*B0 + Bl + B2). Then the 
gradient Grad computed with the half Sobel operator is given by: 
Grad = DifO + Difl 

10 If the signs of DifO and Difl are different, there may be a local extremurn, comprising a 
local minimum or a local maximum, at or near the C pixel. 

The output of step 466 comprises horizontal and vertical gradient images, 
separately obtained from the colorl and color2 images. 

The gradient is then normalized (step 468), preferably with a 

15 normalization operator based on distance from an edge, and preferably separately for 
DifO and Difl. In operation of surface detectors on several materials simultaneously, 
problems arise because each material has a different inherent texture, creating a different 
level of gradients sometimes referred to as "noise". A gradient noise level which 
indicates a defect in one material might be regarded as normal texture in a second 

20 material. 

It is appreciated that various methods might be employed to overcome 
the problem of different texture in different materials. One method would be to apply 
adaptive threshold. Alternatively, a separate normalization function may be provided for 
each material, within one threshold then being applied to the normalized gradients thus 
25 obtained. 
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In step 468 of the present invention, gradient normalization is preferably 
used. For each material, an appropriate normalization function is set in advance, the 
normalization functions being picked to obtain uniform results. Preferably appropriate 
normalization functions may be determined as follows: a histogram of gradients of all 
5 materials is obtained; a median gradient value is determined for each material; and a 
normalization operation, as is well known in the art, is applied to obtain normalization 
functions. 

In step 468, the normalized gradient is then preferably computed as 

follows: 

10 NormGrad = NormFun(material distance from edge) * Gradient * 

EdgeFactor(NarrowArea) 

where: 

NormGrad is the normalized gradient value; 

NormFunction(material distance from edge) is the normalization function 
1 5 for the material, which may be determined as described above; 
Gradient is the input gradient value; 

and EdgeFactor(NarrowArea) is a factor, generally ranging between 0 
and 1, and chosen to give less weight to gradients near an edge; 
EdgeFactor(NarrowArea) preferably has, for example, a value of 0.8 near an edge. 

20 Gradients, whose computation was described above, are analyzed, 

classified, and thresholded (step 470). Reference is now additionally made to Fig. 26, 
which is a simplified flowchart illustration of a preferred implementation of a portion of 
step 470 of Fig. 24, including analysis and classification. In Fig. 26, DifO and Difl are 
preferably as described above with reference to step 466. It is appreciated that, in a case 

25 where a LinearDiff operator is used, as described above, DifO and Difl are preferably 
measured linearly and represent differences between a central pixel and head and tail 
pixels of a vector along the direction (corresponding to the current okdir direction) on 
which LinearDiff is measured. In a case, for example, where c is the gray value of a 
central pixel and b and a are gray values of head and tail pixels, DifO and DifO are 

30 preferably given by: 

DifO = c - b; and 
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Difl = a - c. 

The method of Fig. 26 is self-explanatory. 

Referring back to Fig. 24, in step 470 thresholding is then applied. Based 
on predetermined criteria, typically derived from previous analysis of images, a 
5 threshold is applied to gradient and maxima/minima determinations, preferably to grade 
each gradient or maxima/minima report as follows: 

0 no report 

1 weak report 

2 medium report 
10 3 strong report 

The output of step 470 preferably comprises separate colorl and color2 
reports, for local minima/maxima defects and for gradient defects in each case. The 
colorl and color2 reports are unified (step 472), preferably by applying a "survival of 
the strongest" rule, typically represented by the following logic: 
15 If (report value (colorl) >= report value (color2)) 

report color = colorl 
report value = report(colorl) 
report type = report type (colorl ) 

Else 

20 report color = color2 

report value = report (color2) 
report type = report type (color2) 
Results of step 472 are then filtered (step 474), typically by analyzing a 3 
x 3 spread of reports, computing a weighted sum, and applying a predetermined 
25 threshold; preferably, this operation is performed in a hardware LUT. 

2. Steps relating to the sub resolution channel 404: 
The image is smoothed, preferably with a 5 x 5 chessboard median filter 
(step 476); this step is parallel to step 464 in the full resolution channel. In the sub 
resolution channel, large defects are searched for; hence, it is desirable to ignore small 
30 features, even if they have a large variance from the surrounding area. It is therefore 
preferred to use a large kernel size filter designed to ignore pixels which vary greatly 
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from their neighbors. Preferably, a median filter is used, in which data is sampled and 
the median value is determined and used, as is well-known in the art. For ease of 
implementation, it is desired to perform the median computation on a sub-sample, such 
as a 5 x 5 chessboard sub-sample such as the following, in which pixels marked with an 
X are sampled and those marked with a 0 are not sampled: 
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0 
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X 



Gradient detectors are applied (step 478); this step is parallel to step 466, 
and is preferably similar to step 466 except that a Sobel detector is used within the 
1 0 material. Sobel detectors are described above, particularly in reference to step 466. 

Gradient normalization takes place (step 480), parallel to and preferably 
similar to the gradient normalization of step 468. 

In step 482, gradient thresholding, similar to the thresholding in parallel 
step 470, is performed, and the color reports are unified. Unification of color reports in 
15 step 482 is preferably similar to step 472. A report image, including unfiltered gradient 
defects, is thus produced. 

A large area results filter is applied (step 484); this step is parallel to step 
474. In step 484, preferably a 3 x 3 filter similar to that described above with reference 
to step 474 is first applied. Then, the result of the 3 x 3 filter is used to make a lager 
20 spread, such as a 9 x 9 spread, which is then similarly filtered based on thresholding of a 
weighted sum. Preferably, the 9 x 9 spread is subsampled after filtering, typically 
according to the following pattern, with X representing pixels that are sampled and 0 
representing pixels that are not sampled: 
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Optionally, the result is then diluted when there is a report "cloud" of clustered reports, 
with external reports defining the edge of the cloud not being diluted, while internal 
reports are diluted, thus saving storage space by keep a smaller number of representative 
reports. 

3. Merging of the sub resolution channel report and the full resolution 

channel report. 

The sub resolution channel report and the full resolution channel report 
are merged (step 468). Merging is done by the following rules: 

1. "Survival of the strongest", similar to that described above with 
reference to step 472. 

2. When two reports have the same strength, that of the finer detector is 
used, according to the following order: 

local minima/maxima > full gradient > sub gradient 
In a typical implementation, each item in the final report comprises a 9 bit word, with a 
typical structure as follows: 

Field Bits Values / interpretation 

Report type 2 0 sub gradient 

1 full gradient 

2 local maximum 
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Report strength 

Report color 
Wide/Narrow 
Report material 



3 local minimum 

0 no report 

1 weak report 

2 strong report 

3 diluted report 

0 intensity oriented (colorl) 

1 chromatic oriented (color2) 

0 wide area (within material) 

1 narrow area (close to material edge) 
0-7 (based on color classification) 



Morphological Skeleton Production 

Reference is now made to Fig. 27, which is a simplified flowchart 
illustration of a preferred method of operation of the line and space skeleton subunit 230 
of Fig. 1 . The method of Fig. 27 comprises an erosion method, erosion methods in 
general being well known in the art. The line and space skeleton subunit 230 preferably 
receives as an input binary eels produced by binarization and eel production subunit 
200, preferably derived from monochrome image data. The steps of Fig. 27 preferably 
comprise the following: 

The image undergoing erosion for skeleton production pixel by pixel is 
traversed in predetermined X and Y scanning directions (step 490), such as, for 
example, left to right along each row in the X direction followed by top to bottom in the 
Y direction. 

For each pixel, erosion is performed, typically according to the 4 or 8 
nearest neighbors of the pixel being eroded (step 492). Reference is now additionally 
made to Figs. 28A and 28B, which are simplified pictorial illustrations of an array of 
pixels, before and after partial erosion, useful in understanding the method of Fig. 27. In 
Fig. 28A a pixel being eroded 496, marked as A4, may be eroded in accordance with 
pixel values of either: 

four horizontal and vertical neighbors 498, marked as Al, A3, A5, and 

A7; or 
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eight nearest neighbors comprising the four horizontal and vertical 
neighbors 496 and four diagonal neighbors 500, marked as AO, A2, A6, and A8. 

Which nearest neighbors are chosen may vary according to the analysis 
being performed. Persons skilled in the art will appreciate that erosion based on four 
5 nearest neighbors favors the principal horizontal and vertical axes, while erosion based 
on eight nearest neighbors allows erosion along the diagonals as well. In principle, in 
order to allow more uniform erosion it would be preferable to erode both using four and 
using eight nearest neighbors, but in practice either one of four or eight nearest 
neighbors may be used. In either case, erosion is preferably from black pixels taken to 
10 have the value of 1 to white pixels taken to have the value of 0, it being appreciated that 
erosion from white to black and/or assigning of opposite values from those mentioned 
above to white and black pixels is also possible. Preferred erosion rules are as follows: 

for four neighbors, set the central pixel 496 to 0 if any of the four 
horizontal and vertical neighbors 498 has a value of 0; and 
1 5 for eight neighbors, set the central pixel 496 to 0 if any of the four 

horizontal and vertical neighbors 498 or any of the diagonal neighbors 500 has a value 
ofO. 

Preferably but not necessarily in the present invention, connectivity is 
preserved during erosion; that is, connected black pixels should stay connected and not 

20 be "broken" even after erosion. If connectivity is to be preserved, connectivity 
conditions, typically comprising Euler's conditions, which are well known in the art, are 
applied (step 494). Euler's conditions are geometric conditions which are meant to 
preserve connectivity if the sum or score indicated by Euler's conditions is preserved 
before and after erosion. To preserve connectivity, erosion does not take place if Euler's 

25 conditions are violated. 

Referring to Fig. 28B, pixels which have already been eroded 502 are 
indicated as P0, PI, P2, P3. The pixel being eroded 496 is preferably eroded only if the 
conditions mentioned above with respect to step 492 are fulfilled, and Euler's conditions 
are also fulfilled. Reference is now additionally made to Fig. 29, which is a simplified 

30 pictorial depiction of Euler's conditions. In Fig. 29, the number of occurrences of a 
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plurality of patterns 504 is to be added and subtracted as indicated in Fig. 29 in order to 
obtain the sum which should be invariant before and after erosion. 

Preferably, the method of Fig. 27 is implemented in custom hardware for 
efficiency of operation, as described above. It is appreciated that the method of Fig. 27 
may be performed one or a plurality of times for each image, thus performing one or 
more stages of erosion. Preferably, the number of stages of erosion performed varies 
according to the maximum thickness of any feature being eroded, so that, at the end of 
erosion, a skeleton of one pixel width is obtained. Each stage may be performed with 4 
or 8 neighbors, and with or without preserving connectivity, it being appreciated that use 
of 4 neighbors on some erosions and 8 neighbors on other erosions may aid in obtaining 
isotropy of erosion. 

Reference is now made to Figs. 30A - 30C, which are simplified pictorial 
illustrations useful in understanding a preferred method of operation of the morphology 
subunit 240 of Fig. 1. The morphology subunit 240 preferably receives an input from the 
line and space skeleton subunit 230 and an image of homogeneous color populations 
from the material identification 

In Fig. 30A an example of an image before erosion 506 is depicted. In 
Fig. 30B, an example of a result of erosion of black 508 of the image before erosion 506 
of Fig. 30A, is depicted. In Fig. 30C, an example of a result of erosion of white 510 of 
the image before erosion 506, of Fig. 30A is depicted. 

In Figs. 30B and 30C, examples of typical morphological events which 
are to be detected are shown as follows: open ends 512; junctions 514; and a blob 
junction 516. 

Reference is now made to Figs. 31 A - 31C, which are simplified pictorial 
illustrations of operators, useful in understanding Figs. 30A - 30C. In a preferred 
embodiment of the present invention, examples of operators, also known as kernels, 
depicted in Figs. 31 A - 31C may be applied, as is well known in the art to either the 
image of Fig. 30B or the image of Fig. 30C to detect morphological events. In Fig. 31A 
open end kernel operators 518 are depicted. In Fig. 3 IB, junction operators 520 are 
depicted. In Fig. 31C, a blob junction operator 522, which may be used for depicting 
"black" islands surround by white, is depicted. 
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It is readily appreciated that although shown with respect to a black and 
white image, or binary image, originating from the binarization and eel production 
subunit 200, in accordance with a preferred embodiment of the present invention, 
regions and features in the eroded binary images shown in Figs. 3 OB and 30C are 
5 correlated to homogeneous color populations. In this way particular features, such as 
512; junctions 514; and a blob junction 516 may be related to homogeneous color 
populations, and by further correlation as described above, to the materials and material 
combinations represented by the respective homogeneous color populations. 

Reference is now made to Fig. 32, which is a simplified flowchart 
1 0 illustration of a preferred method of color morphology analysis of an image. The method 
of Fig. 32 is described, for purposes of simplicity of description, after the description of 
binary morphology analysis of an image, it being appreciated that the method of Fig. 32 
might be, for example, implemented, at least in part, in the CLRP unit 120 of Fig. 1, in 
which color information is available. It is appreciated that the method of Fig. 32 is 
1 5 generally useful for color morphology analysis of an image, and may therefore be useful 
in other contexts of color image analysis as well, outside the specific scope of the CRLP 
unit 120 of Fig 1 . It will be appreciated by persons skilled in the art that the method of 
Fig. 32 may be used for analysis of any appropriate optical characteristic of an image, 
and in particular that the method of Fig. 32 is not limited to morphology analysis. 
20 The method of Fig. 32 preferably comprises the following steps: 

A color material image is produced using color methods (step 523), 
typically as described above with reference to color processing. The color material 
image is preferably produced by material identification sub-unit 1 70, and each material 
is correlated to a homogeneous color population. It is readily appreciated that production 
25 of the color material image typically significantly reduces the number of different color 
populations relative to the number of populations in the color image data. 

A gap is preferably created between materials (step 524), typically by 
assigning an artificial color value to pixels in each area along each border between 
materials, in order to separate the material areas for further processing. A binary image 
30 is produced from the output of step 524 (step 525), typically using methods, such as 
thresholding, which are well known in the art. Alternatively, the color image may 
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converted to a gray level monochrome image, by any conventional method known in the 
art, and provided to binarization and eel production sub-unit 200 which produces 
therefrom a binary image. 

A skeleton method is performed on the output of step 525 (step 526); 
5 preferably, the method of Fig. 27 is used. Morphological events in the output step 526 
are identified (step 527), preferably as described above with reference to Figs. 30A - 
30C. 

The morphological events are then correlated to homogenous color 
populations, for example by "coloring" the features according the colors of 

10 corresponding locations in the underlying material image produced by step 523 above 
(step 528). In accordance with a preferred embodiment of the present invention, the 
coloring step is carried out by superimposing the eroded skeleton image and the material 
image so that the features in the skeleton image receive colors according to on of the 
colors of the homogeneous color populations in the superimposed material image. 

1 5 Thus, a color morphology image, comprising a morphology map having 

colors from the homogeneous color populations, or materials, assigned to features, is 
produced. 

Nick and Protrusion P rocessing 
Reference is now made to Figs. 33A and 33B, which, taken together, 

20 comprise a simplified flowchart illustration of a preferred method of operation of the 
nicks and protrusions subunit 220 of Fig. 1. The terms "nick" and "protrusion" refer to 
small defects on an edge; a nick is a defect that extends into the edge, while a protrusion 
is a defect wherein the edge extends outward from its proper location. The nicks and 
protrusions subunit 220 of Fig. 1 is preferably operative to identify nicks and 

25 protrusions and to distinguish nicks and protrusions from normal features of an object 
whose image is being analyzed such as, for example, corners. It is appreciated that 
detection of nicks and protrusions is especially interesting in inspecting objects such as 
described herein, in which a multiplicity of small features having straight edges is 
present, and in which defects such as nicks and protrusions are fairly common, and 

30 typically not desirable. 
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Reference is now additionally made to Fig. 35, which is a simplified 
pictorial illustration of a portion of an image to be analyzed, useful for understanding 
the method of Figs. 33A and 33B. The image of Fig. 35 includes a nick 800, a 
protrusion 810, a corner-in 815 and two corners-out 820, all falling along an edge 830. 
In a preferred embodiment of the present invention, nick and protrusion detection is 
based on analyzing edge contours which define borders between different materials, 
preferably by measuring an aperture of deviation among neighboring contour elements, 
and classifying suspected defects by a change in an angle of a normal to the edge 830, as 
one travels along the edge 830. 

It will be appreciated that nicks and protrusions have opposite profiles of 
angle change of the normal, such that the nick 800 might be characterized as having an 
"in-out-in" change, while the protrusion 810 has an "out-in-out" angle change. The 
corner-in 815, on the other hand, has an "in-in-in" angle change, while the corners-out 
820 have an "out" angle change. It is therefore appreciated than an appropriate method 
of measuring and analyzing normal angle changes should be capable of distinguishing 
between nicks, protrusions, and normal features such as corners. 

The method of Figs. 33A and 33B preferably includes the following 

steps: 

It is appreciated that, for the sake of efficiency of operation, steps 840 
and 930 (both described below) are preferably implemented in parallel, each of steps 
840 and 930 receiving eels to be analyzed as input. 

Break detection is performed for each eel being analyzed in an image, 
typically for all eels in the image (step 840). Step 840 preferably comprises the 
following steps 850, 860, 870, 880, 890, 900, and 910: 

At a cell being analyzed, the eel length and eel directed normal angle are 
measured (step 850). Referring back to Figs. 9 - 1 1, it will be appreciated that there are 
preferably only a small predetermined number of possible eels, and therefore it may be 
preferred to determine eel length and eel angle from a LUT. 

The following steps 870, 880, 890, 900, and 910 are then preferably 
performed for each neighbor eel of the eel being analyzed (step 860). It is appreciated 
that each eel has only 4 neighbors since, referring again back to Figs. 9 - 11, it is 
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appreciated a eel can only continue in the direction of one of the 4 neighbors and not in 
one of the 4 "comer" directions. 

The length of the neighbor eel is checked to make sure that the neighbor 
eel length is above a predefined threshold such as, for example, a threshold of Va pixel 
(step 870). If the neighbor eel length is not above the predefined threshold, processing 
continues with the next neighbor eel (step 880). 

If the neighbor eel length is above the predefined threshold, neighbor eel 
directed normal angles are determined, preferably using a method similar to that 
described above with reference to step 850, and a break angle is computed (step 890). 
Reference is now additionally made to Fig. 34, which is a simplified pictorial 
illustration helpful in understanding step 890 of Fig. 33 A. Fig 34 depicts a eel 891 and 
neighbor eels 892 and 893. Normals 894 and 895 to eels 892 and 893 respectively are 
shown. A break angle 896 is then the break angle of the eel 891 . 

A check is made as to whether the break angle is larger than a 
predetermined threshold such as, for example, a threshold of approximately 22 degrees, 
(step 900). If the break angle is not larger than the predetermined threshold, processing 
continues with step 880, referred to above. 

If the break angle is larger than the predetermined threshold, a break is 
marked for the eel being analyzed (step 910), typically in a suitable map. 

In order to make sure that the region of eels around the eel being 
analyzed in step 840 is marked as having a problem, a break map spread is created ,thus 
effectively expanding the break report; that is, two neighboring pixels, comprising two 
pixels in which the eel being analyzed continues, are also marked as having a break 
(step 920). In expanding the break report, preferably no previous report recorded for a 
neighboring pixel is overwritten, in order prevent a reported break from being 
overwritten by the expansion of a neighboring break. 

Break reports are then extracted (step 925) and passed on to step 980, 
described below. 

Preferably in parallel to the above-described steps of Fig. 33A, feature 
characterization is performed for each eel being analyzed in an image, typically for all 
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eels in the image (step 930). Step 930 preferably comprises the following steps 940, 
950, 960, and 970: 

A midpoint of the eel is determined (step 940). It will be appreciated, as 
described above, that since there are only a limited number of possible eels in each 
5 virtual eel pixel, it may be preferred and most efficient to find the eel midpoint via a 
LUT. 

Two edge eels for the feature which the eel currently being analyzed is 
part of are determined within a spread around the eel being analyzed, typically within a 
5 by 5 array of virtual eel pixels centering around the eel being analyzed (step 950). It is 
10 appreciated that, for regions of efficiency of implementation, it would be desirable to 
determine the feature depth and aperture from the edge eels via a LUT, but a 
straightforward LUT implementation using a 5 by 5 array would required 25 bits of LUT 
address, or 2 25 entries, which is considered to be number of entries which is too difficult 
to process in practice. 

15 A preferred method for finding the edge eels in step 950 comprises the 

following. Reference is now additionally made to Fig. 36, which is a simplified pictorial 
illustration of an array of eel virtual pixels, useful in understanding step 950 of Fig. 33B. 
In the example of Fig. 36, a eel currently being analyzed or central eel 951 is known. A 
first shoulder eel 952 and second shoulder eel 953 are to be determined. 
20 Preferably, 3x3 connectivity analysis is first performed on nine central 

virtual eel pixels 954, comprising the virtual pixel of the central eel 951 and the eight 
first order neighbors thereof The purpose of the connectivity analysis is to assign a type 
to the contour being inspected, typically in accordance with the following: 

Connectivity type Connectivity code 

Main diagonal 0 

Secondary diagonal 1 

Vertical 2 

Horizontal 3 

Upper left 4 

Upper right 5 

Bottom left 6 
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Bottom right 7 
No connectivity 8 
Defective contour 1 5 (OFx) 

Reference is now made to Figs. 37 A and 37B, which comprise simplified 
pictorial illustrations of examples of contours, helpful in understanding the above table. 
Fig. 37A comprises an example of a defective contour, which is generally determined to 
exist when more than one contour is found. Fig. 37B comprises an example of an upper 
5 left contour, similar to that depicted in Fig. 36. 

Preferably, analysis of connected components, such as connected eels, of 
the 16 edge virtual eel pixels 955 is performed. Referring back to the example of Fig. 
36, first and second contours are, for example, found in the 16 edge virtual eel pixels 
955. The following logic is then preferably applied, where n represents the number of 
10 distinct contours found in the 16 edge virtual eel pixels 955: 



n Operation 

0 or 1 unconnected contour found, do not report 

2 or 3 proceed as described below 

> 3 report defective contour (space violation or nick/protrusion( 



Next, two contours of the contours previously found are selected as most 
likely to be connected together, preferably based on distance measures, so that the two 
1 5 contours closest to the central contour located in the nine central virtual eel pixels 954 
are preferably used. 

Preferably, the two eels of the two selected contours which are closest to 
each other, according to Euclidean distance, are found and reported as the shoulders of 
the suspected contour. For example, the first shoulder eel 952 and the second shoulder 
20 eel 953 in Fig. 36 are determined by the above-described method. 

Referring back to Fig. 33B, three characterizing points, based on the two 
edge eels determined in step 950 and the midpoint determined in step 940, are, in a 
preferred embodiment of the present invention, transformed for simplicity of further 
processing (step 960). It is appreciated that, in another embodiment of the present 
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invention, step 960 may be omitted, since step 960 is present primarily to aid in efficient 
computation. It is appreciated that the central eel may be characterized by 4 bits of 
information, while the two edge eels can each be characterized by 4 bits of location 
information and 4 bits of eel address information, for a total of 20 bits of information. 
5 By performing a rigid transformation to the three points, shifting the central eel 
midpoint to the center of the central eel virtual pixel or to another appropriate fixed 
point whose location is known a priori, less data is needed. Typically, 4 bits of 
information may be omitted, thus requiring only 16 bits of information. 

Feature depth and aperture are now determined based on the transformed 

10 points as transformed in step 960 (step 970). Preferably, the depth and aperture are 
determined via a LUT based on the 16 bits of information described above with 
reference to step 960, based on a depth measurement from the bottom right corner of the 
central eel virtual pixel to a line connecting the first and last eels. Reference is now 
additionally made to Fig. 38, which is a simplified pictorial illustration of a typical 

1 5 example of a protrusion, showing a depth 971 and aperture 972 thereof. 

Reference is now made to Fig. 39A, which is a simplified pictorial 
illustration of an array of virtual eel pixels, useful in understanding step 980 of Fig. 33B. 
The array of Fig. 39A comprises an inspected contour 981, having a central virtual eel 
pixel 982, a first shoulder 983, and a last shoulder 984. Reference is additionally made 

20 to Fig. 39B, which is a simplified pictorial illustration of the array of virtual eel pixels of 
Fig. 39A, showing locations of "in" and "out" break reports, denoted by "1" and "O" 
respectively, associated with analysis of the inspected contour 981. Preferably, only 
break reports 985, 986, and 987, corresponding respectively to the central pixel 982, the 
first shoulder 983, and the last shoulder 984, are input into step 980, described below. 

25 Step 980 preferably receives output both from step 925 and from step 

930, both described above. Preferably, in step 980 a result is determined based on the 
input received from steps 925 and 930 by implementing a decision table, typically 
comprising a LUT. The result of step 980 is preferably determined from an appropriate 
LUT. Typically, input to the LUT includes contour depth and aperture; and breaks 

30 recorded in the center of the contour and on the shoulders thereof, also referred to as 
central, first, and last breaks respectively. The results of the LUT typically comprise one 
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of the following: nick; protrusion; corner; no report. The LUT implements an 
appropriate method, typically comprising a brute force decision method, comprising a 
series of logical operations leading to a decision. Preferred logical operations may be 
described in heuristics such as, for example: 
5 a break sequence in-in-in indicates a comer but may be reported as a 

protrusion if its depth greatly exceeds a depth threshold such as, for example, more than 
twice a depth threshold; 

a break sequence in-out-in indicates a nick and will be reported as a nick 
if a minimal nick depth threshold is exceeded ;and 
1 0 a break sequence of none-out-in is reported (where the term "none" refers 

to a case where no break is reported) only if its depth is great enough and its width is 
narrow enough. 

Typically, depth thresholds are specific to the application or to the type 
of object being examined, and might typically vary between 3 /4 of a pixel and 2 or 3 

1 5 pixels, while aperture threshold are typically 2.5 pixels in width. 

The results of step 980 are preferably reported, the results comprising the 
report of nick/protrusion detection, and may be post processed to produce a report, 
typically comprising a trigger for such post processing. The results, as described above, 
may typically comprise: nick; protrusion; comer; or no report. 

20 Reference is now made to Figs. 39C and 39D, which are simplified 

flowchart illustrations of methods useful in conjunction with the system of Fig. 1. The 
method of Fig. 39C comprises a method for producing a color morphology map from a 
color image. The method of Fig. 39D comprises a method for determining color eels in a 
color image. The methods of Figs. 39C and 39D are self-explanatory in light of the 

25 above description. 

Reference is now made to Fig. 40, which is a simplified pictorial 
illustration of an example of a patterned article such as an electrical circuit, useful for 
understanding the operation of the present invention. The apparatus of Fig. 40 is 
intended to provide a particular example of an object which may be analyzed using the 

30 present invention. The example of Fig. 40 is not meant to be limiting. 
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The apparatus of Fig. 40 comprises an electrical circuit 1000, comprising 
in the example of Fig. 40 a ball grid array (BGA) substrate. The BGA 1000, as is well 
known in the art, typically comprises: regions of copper conductor, regions of copper 
conductor which are covered by a partially transparent or translucent coating such as a 
5 solder mask 1004, regions of bare substrate 1006, regions of substrate covered by 
partially transparent or translucent solder mask 1008, regions of copper conductor 
coated with a metal plating such as gold, for example balls 1010 and power lines 1012. 
Additionally, some of the regions plated with gold may also be covered by partially 
transparent or translucent solder mask, such as covered balls 1014. 

10 It is appreciated that each of the regions of different materials and 

materials combinations described hereinabove are characterized by a different 
distinguishing color population. For example, regions of bare copper and gold plated 
copper are distinguishable by color. Solder-mask is typically green, and copper covered 
by solder mask will typically be light green, gold plated copper covered by solder mask 

15 will be very light green, and substrate covered by solder mask will be dark green. In the 
preferred embodiments of the invention described hereinabove, each of the different 
populations can be identified, and binary eels, color eels and color morphology features 
can be produced to represent each of the regions. 

It will also be appreciated by persons skilled in the art that a patterned 

20 article as a BGA may be analyzed, including being analyzed for various surface defects 
such as, for example, scratches 1016 and surface defects 1018, for example stains, pits, 
and other defects, using the present invention. 

The inventors have found that it may be desirable to further process some 
regions of a BGA in an external processing step, regardless of whether a defect is 

25 detected in that area. For example, some regions may be defined by a user, and some 
regions may be associated with particular features. Thus, for example, in some regions 
of interest at particular locations, such as regions of power-lines or balls, a snapshot 
image is preferably generated, and provided for additional image processing, regardless 
of whether any particular feature or defect is found in that region. Additionally and 

30 alternatively, upon the detection of certain features, for example an open end in a gold 
plated copper connector, regardless of where located on the BGA, a snapshot image is 
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preferably generated, and provided for additional image processing, regardless of 
location or whether any particular defect is found with respect to the feature. 

In order to identify features, a eel map may be generated, a part of which 
is shown in Fig. 40, which may be used for further processing. The eel map may identify 
5 both contours, and correlate each contour to the existence of a material on either side 
thereof, for example gold plated copper on one side and substrate on the other side, as 
would be the result for power lines 1012. The eels are preferably operated on directly to 
detect nicks 1020 and protrusions 1022. 

Using the present invention, a morphology map is also generated, a part 
10 of which is shown in Fig. 40, in which a morphology skeleton is produced for the object. 
In the morphology map, a skeleton 1024 of the features is generated, and the skeleton 
1024 is preferably correlated to a particular material. Thus, for example, features on 
skeleton 1024, which represents a region of power line 1024 would be correlated to gold 
plated copper. 

15 Referring back to Fig. 1, as described above the image analysis system 

1 00 is preferably operative to analyze an input image and to pass results of the analysis 
on for further processing. Preferably, as further described in detail above with reference 
to Figs. 1 - 40 for the various components of the image analysis system 100, results of 
the analysis which are made available for further processing include: 

20 1. information characterizing defects: surface defects; nick and 

protrusion defects; and defective color contour elements (eels). 

2. information characterizing the article being expected: binary eels, a 
skeleton, skeleton features, a color skeleton, and color eels. 

3. a snapshot, typically comprising a color RGB snapshot, of an area 
25 around each region of interest, for example around each predetermined type of feature, 

such as power lines 1012 and balls 1010 (Fig. 40), each user defined area, and each 
defect. 

Without limiting the generality of the foregoing, each of the lists of 
results immediately above are typically output in separate data channels for further 
30 processing as desired. Preferably, the separate data channels comprise two physical 
channels as follows: 
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A first channel preferably includes the following reports and data: 
A report of binary eels; 
a report of color eels; 

a report of morphological and skeleton features; and 

a report of defects such as nicks and protrusions. 

A second channel preferably includes the following reports and data: 

A color defect report indicating reports of color surface defects from the 
COMRADD 190 and gray surface defects from the CABS unit 130; and 

a snapshot report including snapshots of regions of interest, for example, 
each region of reported defect, each morphological feature of interest and each user 
defined region of interest. 

It is appreciated that various features of the invention . which are, for 
clarity, described in the contexts of separate embodiments may also be provided in 
combination in a single embodiment. Conversely, various features of the invention 
which are, for brevity, described in the context of a single embodiment may also be 
provided separately or in any suitable subcombination. 

It will be appreciated by persons skilled in the art that the present 
invention is not limited by what has been particularly shown and described hereinabove. 
Rather the scope of the invention is defined only by the claims which follow: 
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What is claimed is: 

CLAIMS 

1 . A method for determining a location of a border in a color image, the 

image comprising at least two color populations, between a first color region associated 
5 with a first one of the two color populations and a second color region associated with a 
second one of the two color populations, both the first color region and the second color 
region being comprised in the color image, the method comprising: 

identifying an approximate border location between said first color 
region and said second color region; 
10 determining a plurality of candidate border locations between the first 

color region and the second color region, each of said plurality of candidate border 
locations being determined by applying a corresponding border location method chosen 
from among a plurality of border location methods; 

choosing one method from among the plurality of border location 
1 5 methods as a preferred method; and 

determining a location of a border between said first color region and 
said second color region by designating one of said plurality of candidate border 
locations associated with the preferred method as the border. 

20 2. A method according to claim 1 and wherein said color image comprises 

an image of a patterned article. 

3. A method according to claim 2 and wherein said color image comprises a 
color image of an electrical circuit. 

25 

4. A method according to claim 3 and wherein said color image comprises a 
color image of a lead frame. 

5. A method according to claim 1 and wherein the preferred border location 
30 method is chosen based on a rule for choosing a preferred border location method for a 

predetermined combination of color populations. 
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6. A method according to claim 1 and wherein the preferred border location 
method is chosen based on comparison of the border location result from applying at 
least two border location methods. 

5 

7. A method for determining a location of a border in a color image, the 
image comprising at least two color populations, between a first color region associated 
with a first one of the two color populations and a second color region associated with a 
second one of the two color populations, both the first color region and the second color 

10 region being comprised in the color image, the method comprising: 

providing a border analysis map comprising, for each distinct pair of 

color populations, an indication of a preferred method for identifying the location of a 

border between said color populations; 

identifying an approximate border location between said first color 
1 5 region and said second color region; and 

determining a location of a border between said first color region and 

said second color region by using the preferred method indicated, in the border analysis 

map, for identifying the location of a border between said first color population and said 

second color population. 

20 

8 . A method according to claim 7 and wherein said color image comprises 
an image of a patterned article. 

9. A method according to claim 8 and wherein said color image comprises a 
25 color image of an electrical circuit. 

10. A method according to claim 8 and wherein said color image comprises a 
color image of a lead frame. 

30 11. A method according to claim 9 and wherein said electrical circuit 

comprises a printed circuit board. 
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12 - A method according to claim 7 and wherein the color image comprises a 

color image of an object, and 

each said color population is associated with one material of a plurality 
5 of available materials 



13 ■ A method according to claim 7 and wherein said color image also 

comprises a third color region associated with a third color population, and 

said identifying step comprises, for each two color regions among the 
1 0 first color region, the second color region, and the third color region which border each 
other, identifying an approximate border location between said two color regions, and 

said determining step comprises, for each said two color regions, 
determining a location of a border between said two color regions by using the preferred 
method indicated, in the border analysis map, for identifying the location of a border 
1 5 between the two color populations associated with said two color regions. 

14 - A method according to claim 13 and also comprising: 

defining a window having a window size; 

examining a portion of said color image falling within said window to 
20 determine the number of distinct color regions at least a part of which are included in 
said portion; and 

if the number of distinct color regions is greater than two, classifying all 
borders within the window as belonging to a junction and determining a location of each 
border within said window using a preferred method for determining the location of a 
25 border belonging to a junction. 



15 - A method according to claim 14 and wherein the preferred method for 

determining the location of a border belonging to a junction comprises a 
high-water-mark method. 

30 
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16. A method according to claim 7 and wherein the color image is 

represented using a color image representation method having a plurality of color 
defining characteristics, and 

each said preferred method is selected from a plurality of available 

5 methods, and 

the plurality of available methods includes, for each color defining 
characteristic within the plurality of color defining characteristics, a method comprising 
performing an edge locating method in a component of the color image associated with 
said color defining characteristic. 

10 

1 7 - A method according to claim 16 and wherein said performing an edge 

locating method in a component comprises performing a sub-pixel contour-element 
locating method. 

15 18. A method according to claim 16 and wherein said performing an edge 

locating method only in a component comprises performing a high-water-mark method. 

19. A method according to claim 16 and wherein said color image 
representation method comprises an RGB method and said plurality of color defining 

20 characteristics comprises an R component, a G component, and a B component. 

20. A method according to claim 16 and wherein said color image 
representation method comprises an HSI method and said plurality of color defining 
characteristics comprises an H component, and S component, and an I component. 

25 

21- A method for producing a color morphology map from a color image, the 

method comprising: 

providing a color image; 

reducing the color image to produce a reduced image comprising a 
30 plurality of color populations; 
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determining borders between adjacent ones of the plurality of color 
populations in the reduced image; 

segmenting the reduced image to produce a binary image comprising 
borders and at least one non-border region; 
5 producing a skeleton image of the at least one non-border region, said 

skeleton image comprising a plurality of portions; and 

assigning to each portion of the skeleton a color identity corresponding to 
one of the color populations. 

10 22. A method according to claim 21 and wherein the assigning step includes 

assigning a color population as a function of the spatial location of a portion of the 
skeleton relative to the color population. 

23. A method according to claim 22 and wherein the assigning step further 
1 5 includes superimposing the skeleton image on the reduced image. 

24. A method according to claim 21 and wherein the color image comprises 
a 24 bit color image. 

20 25. A method according to claim 21 and wherein the reduced image 

comprises a 3 bit image, and 

the plurality of color populations comprises at most 8 color populations. 

26. A method according to claim 21 and wherein said determining step 
25 comprises determining borders to an accuracy of one pixel. 

27. A method according to claim 21 and also comprising: 

identifying the presence of at least one morphological feature based, at 
least in part, on a result of the method. 

30 
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28. A method according to any of claims 21-27 and wherein the color 
image comprises an image of an patterned article. 

29. A method according to claim 28 and wherein the patterned article 
comprises an electrical circuit. 

30- A method for determining color contour elements (eels) in a color image, 
the method comprising: 

providing a color image; 

reducing the color image to produce a reduced image comprising a 
plurality of color populations; 

determining borders between adjacent ones of the plurality of color 
populations, said borders comprising a plurality of border segments each border segment 
being surrounded by two color populations; and 

for each border segment, assigning a value to the border segment based, 
at least in part, on the two color populations surrounding the border segment. 

31- A method according to claim 30 and wherein the color image comprises 
a 24 bit color image. 

32. A method according to claim 30 and wherein the reduced image 
comprises a 3 bit image. 

33. A method according to claim 30 and wherein said determining step 
comprises determining borders to an accuracy of one pixel. 

34. A method according to claim 30 and wherein said determining step 
comprises determining borders to an accuracy of less than one pixel. 

35. A method according to claim 30 and wherein each said border segment 
has a first side and a second side, and 
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said assigning step includes assigning a value based, at least in part, on 
the color population on the first side of said border segment and on the color population 
on the second side of said border segment. 

5 36. A method according to any of claims 30 - 35 and wherein the color 

image comprises an image of an patterned article. 

37. A method according to claim 36 and wherein the patterned article 
comprises an electrical circuit. 

10 

38. A method for identifying irregularities in a smooth curve in an image, the 
method comprising: 

identifying an ordered plurality of points along the smooth curve, each of 
said ordered plurality of points having at least one neighboring point in the ordered 
15 plurality of points; 

determining a normal direction to the smooth curve at each of said 
ordered plurality of points; and 

identifying a point of said ordered plurality of points as being part of an 
irregularity based, at least in part, on a local normal difference comprising a difference 
20 between the normal direction to the smooth curve at said point and a normal direction to 
the smooth curve at said at least one neighboring point of said point. 

39. A method according to claim 38 and wherein the image comprises an 
image of a patterned article. 

25 

40. A method according to claim 39 and wherein said image comprises an 
image of an electrical circuit. 

43. A method according to claim 38 and wherein said local normal 

30 difference comprises a difference in direction of more than a predetermined angle. 
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42. A method according to claim 41 and wherein said predetermined angle 
comprises an angle of approximately 22 degrees. 

43. A method according to claim 38 and wherein said image comprises a 
5 multiplicity of pixels and each of said ordered plurality of points is associated with one 

pixel. 

44. A method according to claim 43 and wherein each pixel comprised in 
said smooth curve is associated with one of said ordered plurality of points. 

10 

45. A method according to claim 43 and wherein only some pixels 
comprised in said smooth curve are associated with one of said ordered plurality of 
points. 

1 5 46. A method according to claim 45 and wherein said some pixels are chosen 

so that a fixed number of pixels not associated with one of said ordered plurality of 
points falls between each one of said ordered plurality of points and each said at least 
one neighbor of each one of said ordered plurality of points. 

20 47. A method according to claim 38 and also comprising: 

determining a feature depth and feature aperture for said irregularity; and 
classifying said feature based, at least in part, on a result of said 
determining step. 

25 48. A method according to claim 47 and where said classifying includes 

classifying into one of a predetermined set of categories comprising at least one of the 
following categories: 
a nick; 

a protrusion; and 
30 a contour. 
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49. A method for detecting an irregularity in an electrical circuit, the method 
comprising identifying an irregularity in a smooth curve representing a border between 
two materials in an image of the electrical circuit in accordance with a method 
according to any of claims 38 - 48 

5 

50. A method for classifying an irregularity in an image of an object, the 
method comprising: 

determining a feature depth and feature aperture for said irregularity; and 
classifying said feature based, at least in part, on a result of said 
10 determining step. 

51. A method according to claim 50 and wherein said object comprises an 
electrical circuit. 

52. A method according to claim 50 and where said classifying includes 
classifying into one of a predetermined set of categories comprising at least one of the 
following categories: 

a nick; 

a protrusion; and 
a contour. 

53. A method for identifying a defect in a portion of an image of an object, 
the portion being close to a location in the image identified as an edge, the method 
comprising: 

25 identifying an excluded direction associated with the edge; and 

searching for defects, close to the location identified as an edge, in a 
plurality of directions not including the excluded direction. 

54. A method according to claim 53 and wherein said object comprises an 
30 electrical circuit. 
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55. A method according to claim 53 and wherein the excluded direction 
comprises a direction perpendicular to the edge. 

56. A method according to claim 53 and wherein said defect comprises a 
5 scratch. 

57. A method according to claim 53 and wherein said defect comprises a 
stain. 

10 58. A method for identifying a surface defect in an image of an object, the 

image comprising a multiplicity of pixels, the method comprising: 

choosing a region in which a surface defect is to be identified; 
subsampling the region thereby producing a subsampled image of the 

region; 

15 analyzing the subsampled image and identifying therefrom a surface 

defect in the region. 

59. A method according to claim 58 and wherein said image comprises a 
color image of an electrical circuit. 

20 

60. A method according to claim 58 and wherein said subsampling 
comprises subsampling non-adjacent pixels. 

61- A method according to claim 60 and wherein separation of non-adjacent 

25 pixels respective of adjacent pixels comprises a subsampling ratio. 

62. A method according to claim 61 and wherein the subsampling ratio is 

between approximately 1:13 and approximately 1:49. 
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63. A method according to claim 58 and wherein said subsampling 

comprises subsampling in accordance with an approximate average distance between 
subsampled pixels. 

5 64. A method according to claim 63 and wherein said approximate average 

distance between subsampled pixels is between approximately 5 pixels and 
approximately 1 1 pixels. 

65. A method according to claim 58 and wherein the subsampling step 
10 comprises subsampling a portion of the region, and 

the subsampling step is performed a plurality of times using a plurality of 
portions of the region. 

66. A method according to claim 65 and wherein at least two of the plurality 
1 5 of portions of the region comprise partially overlapping portions of the region. 

67. A method for applying an image processing operator to a pixel in an 
array of pixels to produce a value, the method comprising: 

providing an n x n spread of a central pixel; 
20 if the central pixel comprises an edge pixel, assigning the value 0 to a 

result; 

if the central pixel does not comprise an edge pixel, performing the 
following steps for each one pixel in the n x n spread: 

if the one pixel is not an edge pixel, choosing the one pixel as a 

25 chosen pixel; 

if the one pixel is an edge pixel and a mirror image of the one 
pixel within the n x n spread is not an edge pixel, choosing the mirror image as the 
chosen pixel, otherwise choosing the central pixel as the chosen pixel; and 

adding to the sum a product of the value of the chosen pixel and 
30 the result of applying a Gaussian operator to the central pixel; and 

dividing the sum by the square of (n+1) to produce a result. 
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68. A method according to claim 67 and wherein n is equal to 3. 

69. A method for computing a gradient with a convoluted operator 
5 comprising: 

providing an image processing operator in the form of an n x n array of 
values, wherein n is an odd integer and the central value of the n x n array may take on 
either a positive value or a negative value; 

applying the image processing operator with the central value having a 
10 positive value to produce a first intermediate result; 

applying the image processing operator with the central value having a 
negative value to produce a second intermediate result; and 

summing the first intermediate result and the second intermediate result 
to produce a final result. 

15 

70. A method according to claim 69 and also comprising: 

comparing the arithmetic sign of the first intermediate result and the 
arithmetic sign of the second intermediate result; and 

determining the presence of a local extremum based on a result of the 
20 comparing step. 



71. A method according to claim 70 and wherein said determining comprises 

determining with less than complete certainty. 

25 72. A method according to claim 69 and wherein n is equal to 5. 

73. An automated optical inspection device suitable for inspection of 



patterned articles comprising: 

at least one detector providing a polychromatic image output of at least a 
30 portion of a patterned article having a plurality of elements; and 

processing circuitry receiving said polychromatic image output and 
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providing at least one of the following inspection functionalities: 

inspection of a metal coating on at least one of the elements in the 
plurality of elements; and 

inspection of an at least partially transparent coating on at least 
5 one element in the plurality of elements. 

74. An automated optical inspection device suitable for inspection of 
patterned articles comprising: 

at least one detector providing a polychromatic image output of at least a 
1 0 portion of a patterned article comprising a plurality of elements; and 

processing circuitry receiving said polychromatic image output and 
providing the following inspection functionalities: 

inspection of a metal coating on at least one element in the 
plurality of elements; and 
1 5 inspection of an at least partially transparent coating on at least 

one element in the plurality of elements. 

75. An automated optical inspection device according to claim 73 and 
wherein said processing circuitry receiving said polychromatic image output provides 

20 the following additional inspection functionality: 

detection of residues on a surface of said patterned article. 

76. An automated optical inspection device according to claim 74 and 
wherein said processing circuitry receiving said polychromatic image output provides 

25 the following additional inspection functionality: 

detection of residues on a surface of said patterned article. 

77. An automated optical inspection device according to claim 73 and 
wherein said functionality of inspection of a metal coating on at least one of the 

30 elements in the plurality of elements comprises at least one of the following inspection 
modalities: 
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detection of voids in the metal coating; 

detection of surface irregularities in the metal coating; 

detection of scratches in the metal coating; 

detection of the presence of foreign materials on the metal coating; 
5 detection of stains on the metal coating; 

detection of oxidation of the metal coating; and 
detection of the shape of the metal coating. 

78. An automated optical inspection device according to claim 74 and 
10 wherein said functionality of inspection of a metal coating on at least one of the 

elements in the plurality of elements comprises at least one of the following inspection 
modalities: 

detection of voids in the metal coating; 
detection of surface irregularities in the metal coating; 
15 detection of scratches in the metal coating; 

detection of the presence of foreign materials on the metal coating; 
detection of stains on the metal coating; 
detection of oxidation of the metal coating; and 
detection of the shape of the metal coating. 

20 

79. An automated optical inspection device according to claim 73 and 
wherein said functionality of inspection of an at least partially transparent coating on at 
least one of the elements in the plurality of elements comprises at least one of the 
following inspection modalities: 

25 detection of voids in the at least partially transparent coating; 

detection of surface irregularities in the at least partially transparent 

coating; 

detection of scratches in the at least partially transparent coating; 
detection of the presence of foreign materials on the at least partially 
30 transparent coating; 

detection of stains on the at least partially transparent coating; and 
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detection of the shape of the at least partially transparent coating. 

80. An automated optical inspection device according to claim 74 and 
wherein said functionality of inspection of an at least partially transparent coating on at 
least one of the elements in the plurality of elements comprises at least one of the 
following inspection modalities: 

detection of voids in the at least partially transparent coating; 

detection of surface irregularities in the at least partially transparent 

coating; 

detection of scratches in the at least partially transparent coating; 
detection of the presence of foreign materials on the at least partially 
transparent coating; 

detection of stains on the at least partially transparent coating; and 
detection of the shape of the at least partially transparent coating. 

81. An automated optical inspection device according to claim 73 and 
wherein said functionality of detection of residues on a surface of said patterned article 
comprises at least one of the following detection modalities: 

detection of surface irregularities on a surface of said patterned article; 
detection of the presence of foreign materials on a surface of said 
patterned article; 

detection of stains on a surface of said patterned article; and 
detection of the shape of marks on a surface of said patterned article. 

82. An automated optical inspection device according to claim 74 and 
wherein said functionality of detection of residues on a surface of said patterned article 
comprises at least one of the following detection modalities: 

detection of surface irregularities on a surface of said patterned article; 
detection of the presence of foreign materials on a surface of said 
patterned article; 

detection of stains on a surface of said patterned article; and 
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detection of the shape of marks on a surface of said patterned article. 

83. An automated optical inspection device according to any of claims 73 - 
82 and wherein the patterned article comprises an electrical circuit. 

5 

84. An automated inspection device according to claim 83 and wherein the 
elements comprise conductors in the electrical circuit. 

85. An automated optical inspection device according to claim 84 and 
10 wherein the electrical circuit comprises a printed circuit board. 

86. An automated optical inspection device according to claim 84 and 
wherein the electrical circuit comprises a ball grid array substrate. 

15 87. An automated optical inspection device according to any of claims 73 - 

82 and wherein the patterned article comprises a lead frame. 

88. An automated optical inspection device according to any of claims 73 - 
87 and wherein the metal coating comprises a metal plating. 

20 

89. An automated optical inspection device according to claim 86 and 
wherein the elements comprise balls in the ball grid array substrate. 

90. An automated optical inspection device according to any of claims 73 - 
25 82 wherein the patterned article comprises an etched metal substrate. 

91. An automated optical inspection device according to any of claims 73 - 
82 and wherein the patterned article comprises an engraved metal substrate. 

30 92. An automated optical inspection device suitable for inspection of an 

article and comprising: 
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at least one detector providing a polychromatic image output of at least a 
portion of an article; and 

processing circuitry receiving said polychromatic image output and 
providing edge detection to sub-pixel accuracy operative to distinguish borders among 
5 regions of different color. 



93. An automated optical inspection device according to claim 92 and 
wherein said processing circuitry is operative to distinguish a border between a first 
color region associated with a first color population and a second color region associated 

1 0 with a second color population by identifying an approximate border location between 
said first color region and said second color region and determining a location of a 
border between said first color region and said second color region by using a preferred 
method for identifying the location of a border between said first color population and 
said second color population. 

15 

94. An automated optical inspection device according to claim 93 and 
wherein said polychromatic image output comprises a color image represented using a 
color image representation method having a plurality of color defining characteristics, 
and 

20 each said preferred method is selected from a plurality of available 

methods, and 

the plurality of available methods includes, for each color defining 
characteristic within the plurality of color defining characteristics, a method comprising 
performing an edge locating method in a component of the color image associated with 
25 said color defining characteristic. 

95. An automated optical inspection device according to claim 94 and 
wherein the edge locating method comprises a sub-pixel contour-element locating 
method. 

30 

96- An automated optical inspection device suitable for inspection of an 
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article and comprising: 

at least one detector providing an image output of at least a portion of an 
article having at least three different regions, each of which regions is distinguished at 
least by an optical characteristic detectable by said detector; and 
5 processing circuitry receiving said image output and providing edge 

detection to sub-pixel accuracy operative to distinguish borders of at least two of said at 
least three different regions. 

97 An automated optical inspection device according to claim 96 and 

10 wherein said processing circuitry receives said image output and provides edge 
detection to sub-pixel accuracy operative to distinguish borders of said at least three 
different regions. 

98. An automated optical inspection device according to claim 96 and 
15 wherein said processing circuitry provides mapping based at least partially on said 

image output which identifies materials in each of said different regions having a 
common border. 

99. An automated optical inspection device suitable for inspection of an 
20 article and comprising: 

at least one detector providing an image output of at least a portion of an 

article; and 

processing circuitry receiving said image output and providing mapping 
based at least partially on said image output which identifies materials in each of 
25 different regions of said at least a portion of said article having a common border. 

100. An automated optical inspection device according to claim 99 and 
wherein said processing circuitry is operative to sub-pixei accuracy. 

30 101. An automated optical inspection device suitable for inspection of an 

article comprising: 
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at least one detector providing a polychromatic image output of at least a 
portion of an article; 

processing circuitry receiving said polychromatic image output and 
providing at least one of the following inspection functionalities: 
5 segmentation of said polychromatic image into a segmented color image 

comprising a plurality of color regions each having one of a plurality of representative 
color possibilities; 

binarization of the segmented color image into a binary image including 
regions and borders; 

10 morphological erosion of the regions to provide a skeleton representing 

the regions; and 

assignment of color regions to elements in the skeleton. 

102. An automated optical inspection device according to claim 101 and 
15 wherein the processing circuitry is operative to superimpose the skeleton and the 

segmented color image, and to assign data representative of regions to elements in the 
skeleton as a function of the superimposition. 

103. An automated optical inspection device according to claim 102 and 
20 wherein the processing circuitry is operative to assign data representative of regions to 

elements in the skeleton the superimposed location of an element of the skeleton in a 
color region. 

104. An automated optical inspection device suitable for inspection of an 
25 article comprising: 

at least one detector providing a polychromatic image output of at least a 
portion of an article; and 

processing circuitry receiving said polychromatic image output and 
operative to segment said polychromatic image into plurality of color regions chosen 
30 from among a predefined number of color region possibilities, and to generate a 
representation of the polychromatic image comprising color region data representing 
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each color region in the image, and border data for representing each border among 
color regions. 

105 - A col ° r image representing a patterned article, the color image 
comprising a morphologically eroded polychromatic image of the patterned article. 

106 - A color image according to claim 105 and wherein colors of portions of 
said morphologically eroded polychromatic image indicate materials at various regions 
of the patterned article. 

107 - A color image according to any of claims 105 - 106 and wherein the 
patterned article comprises an electrical circuit. 

108 - A color image according to claim 107 and wherein the elements 
comprise conductors in the electrical circuit. 

109 - A color image according to claim 108 and wherein the electrical circuit 
comprises a printed circuit board. 

1 i0 - A color image according to claim 108 and wherein the electrical circuit 

comprises a ball grid array substrate. 

lu - A color image according to any of claim 105 - 106 and wherein the 

patterned article comprises a lead frame. 

112 - A color image according to any of claims 105 - 111 and wherein the 

metal coating comprises a metal plating. 

A color image according to claim 110 and wherein the elements 
comprise balls in the ball grid array substrate. 
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114. A color image according to any of claims 105 - 106 wherein the 
patterned article comprises an etched metal substrate. 

115. A color image according to any of claims 105 - 106 and wherein the 
patterned article comprises an engraved metal substrate. 

1 1 6. An automated optical inspection device suitable for inspection of an 



article comprising: 

at least one detector providing a polychromatic image output of at least a 
portion of an article; and 

processing circuitry receiving said polychromatic image output and 
providing a color image representing said article, the color image comprising a 
morphologically eroded polychromatic image of at least part of said article. 

117. An automated optical inspection device according to claim 116 and 

wherein colors of portions of said morphologically eroded polychromatic image indicate 
materials at various regions of the article. 

118- A color image representing a patterned article, the color image 

comprising a map which indicates borders between different regions of an article and 
identifies materials in each of said different regions having a common border. 

1 1 9. A color image according to claim 1 1 8 and wherein colors of portions of 

said map indicate materials at said different regions. 

120 - A color image according to claim 119 and wherein each color comprises 

one color from a predetermined possibility of colors. 

121 • A color image according to any of claims 118 - 120 and wherein the 

patterned article comprises an electrical circuit. 
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122. A color image according to claim 121 and wherein the elements 
comprise conductors in the electrical circuit. 

123. A color image according to claim 122 and wherein the electrical circuit 
5 comprises a printed circuit board. 

124. A color image according to claim 122 and wherein the electrical circuit 
comprises a ball grid array substrate. 

10 125. A color image according to any of claims 118 - 120 and wherein the 

patterned article comprises a lead frame. 

126. A color image according to any of claims 118 - 125 and wherein the 
metal coating comprises a metal plating. 

15 

127. A color image according to claim 124 and wherein the elements 
comprise balls in the ball grid array substrate. 

128. A color image according to any of claims 118 - 120 and wherein the 
20 patterned article comprises an etched metal substrate. 

129. A color image according to any of claims 118 - 120 and wherein the 
patterned article comprises an engraved metal substrate. 

25 130. An automated optical inspection device suitable for inspection of an 

article comprising: 

at least one detector providing a polychromatic image output of at least a 
portion of an article; and 

processing circuitry receiving said polychromatic image output and 
30 providing a color image representing said article, the color image comprising a map 
which indicates borders between different regions of an article and identifies materials 
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in each of said different regions having a common border. 

131. An automated optical inspection device according to claim 130 and 
wherein colors of portions of said map indicate materials at various regions of the 

5 article. 

132. An automated optical inspection device according to claim 130 and 
wherein said processing circuitry receives said image output and provides border 
indication to sub-pixel accuracy. 

10 

133. An automated optical inspection device according to claim 131 and 
wherein each color comprises a color chosen from among a predetermined possibility of 
colors. 

15 134. An automated optical inspection device according to any of claims 130 - 

133 and wherein the patterned article comprises an electrical circuit. 

135. An automated optical inspection device according to claim 134 and 
wherein the electrical circuit comprises a printed circuit board. 

20 

136. An automated optical inspection device according to claim 135 and 
wherein the electrical circuit comprises a ball grid array substrate. 

137. An automated optical inspection device according to any of claims 130 - 
25 133 and wherein the patterned article comprises a lead frame. 

138. An automated optical inspection device according to any of claims 130 - 
137 and wherein the different materials include different metals. 

30 139. An automated optical inspection device according to claim 135 and 

wherein the different regions include uncoated metal conductors, coated metal 
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conductors, uncoated substrate, and coated substrate. 

140. An automated optical inspection device according to any of claims 130 - 
133 and wherein the patterned article comprises an etched metai substrate. 

5 

141. An automated optical inspection device according to any of claims 130 - 133 and 
wherein the patterned article comprises an engraved metal substrate. 

142. An automated optical inspection device suitable for inspection of a 
1 0 patterned article comprising: 

at least one detector providing an image output of at least a portion of a 
patterned article comprising at least three materials having detectable edges; and 

processing circuitry receiving said image output and providing an 
inspection functionality comprising binarization of the image output to provide a binary 
1 5 image including region image data and border image data. 

143. An automated optical inspection device according to claim 142 and 
wherein said patterned article comprises an electrical circuit. 

20 144. An automated optical inspection device according to claim 143 and 

wherein said electrical circuit comprises a printed circuit board. 

145. An automated optical inspection device according to claim 144 and 
wherein said printed circuit board comprises a ball grid array substrate. 

25 

146. An automated optical inspection device according to claim 143 and 
wherein said patterned article comprises a lead frame. 

147. An automated optical inspection device according to claim 143 and 
30 wherein said patterned article comprises an etched metal substrate. 
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148. An automated optical inspection device according to claim 143 and 
wherein the patterned article comprises an engraved metal substrate. 

149. An automated optical inspection device suitable for inspection of a 
5 patterned article comprising: 

at least one detector providing an image output of at least a portion of a 
patterned article comprising a plurality of materials having detectable borders between 
the plurality of materials; and 

processing circuitry receiving said image output and providing an 
10 inspection functionality comprising pattern analysis of image data relating to the borders 
to provide an indication of the presence of nicks and protrusions along said borders. 

150. An automated optical inspection device according to claim 149 and 
wherein said patterned article comprises an electrical circuit. 

15 

151. An automated optical inspection device according to claim 150 and 
wherein said patterned article comprises a printed circuit board. 

152. An automated optical inspection device according to claim 151 and 
20 wherein said printed circuit board comprises a ball grid array substrate. 

153. An automated optical inspection device according to claim 150 and 
wherein said patterned article comprises a lead frame. 

25 154. An automated optical inspection device according to any of claims 149 - 

153 and wherein said plurality of materials comprises at least three materials. 

155- An automated optical inspection device according to claim 154 and 

wherein said materials include materials from the following group: uncoated metal 
30 conductors, coated material conductors, uncoated substrate, coated substrate. 
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156. An automated optical inspection device according to claim 150 and 
wherein said patterned article comprises an etched metal substrate. 

157. An automated optical inspection device according to claim 150 and 
5 wherein the patterned article comprises an engraved metal substrate. 

158. An automated optical inspection device according to any of claims 149 - 
157 and wherein said inspection functionality comprising pattern analysis includes: 

identifying an ordered plurality of points along a border defined by the 
10 image data relating to the borders, each of said ordered plurality of points having at least 
one neighboring point in the ordered plurality of points; 

determining a normal direction to the smooth curve at each of said 
ordered plurality of points; and 

identifying a point of said ordered plurality of points as being part of an 
15 irregularity comprising one of a nick and a protrusion based, at least in part, on a local 
normal difference comprising a difference between the normal direction to the smooth 
curve at said point and a normal direction to the smooth curve at said at least one 
neighboring point of said point. 

20 159. An automated optical inspection device suitable for inspection of an 

article comprising: 

at least one detector providing an image output of at least a portion of an 
article comprising multiple materials having detectable borders between the materials; 
and 

25 processing circuitry receiving said image output and providing at least 

one of the following inspection functionalities: 

segmentation of said image output into a segmented image including 
region image data and border image data; 

analysis of the region image data using a first technique; and 
30 analysis of the border image data using a second technique, different 

from the first technique. 
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160. An automated optical inspection device according to claim 159 and 
wherein said article comprises an electrical circuit. 

161. An automated optical inspection device according to claim 159 and 
wherein said first technique includes neighborhood analysis and wherein said second 
technique includes border following analysis for differentiating border-near pixels from 
non-border-near pixels. 

162. An automated optical inspection device suitable for inspection of an 
article comprising: 

at least one detector providing a polychromatic image output of at least a 
portion of an article comprising multiple materials having detectable borders between 
the materials; and 

processing circuitry receiving said polychromatic image output and 
providing at least one of the following inspection functionalities: 

segmentation of said polychromatic image output into a segmented color 
image including color regions chosen from among a set of predetermined colors; 

analysis of the region image data relating to a first color using a first 

technique; and 

analysis of the region image data relating to a second color using a 
second technique, different from the first technique. 

163. An automated optical inspection device according to claim 162 and 
wherein said article comprises an electrical circuit. 

164. An automated optical inspection device according to claim 163 and 
wherein said electrical circuit comprises a printed circuit board. 

165. An automated optical inspection device according to claim 164 and 
wherein said printed circuit board comprises a ball grid array substrate. 
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166. An automated optical inspection device according to claim 162 and 

wherein said article comprises a lead frame. 

5 167. An automated optical inspection device suitable for inspection of an 

article comprising: 

at least one detector providing an image output of at least a portion of an 
article comprising multiple materials having detectable borders between the materials; 
and 

10 processing circuitry receiving said image output and providing at least 

one of the following inspection functionalities: 

segmentation of said image output into a segmented image including 
region image data and border image data; and 

smoothing the region image data in a manner not influenced by at least 
1 5 some of the border image data. 

168. An automated optical inspection device according to claim 167 and 

wherein said article comprises an electrical circuit. 

20 169. An automated optical inspection device according to claim 168 and 

wherein said electrical circuit comprises a printed circuit board. 

170. An automated optical inspection device according to claim 169 and 
wherein said printed circuit board comprises a ball grid array substrate. 

25 

171. An automated optical inspection device according to claim 167 and 
wherein said article comprises a lead frame. 

172. An automated optical inspection device suitable for inspection of an 
30 electrical circuit comprising: 
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at least one detector providing an image output of at least a portion of an 
electrical circuit comprising multiple materials; and 

processing circuitry receiving said image output and providing an output 
indication of the presence of at least one predetermined material. 

17 3. An automated optical inspection device according to claim 172 and 

wherein said electrical circuit comprises a printed circuit board. 

174 - An automated optical inspection device according to claim 173 and 
wherein said printed circuit board comprises a ball grid array substrate. 

175 - An automated optical inspection device according to claim 174 and 
wherein said electrical circuit comprises a flat panel display substrate. 

176. An automated optical inspection device according to claim 172 and 
wherein said processing circuitry provides an output indication of the presence of at 
least two materials out of a group of at least three predetermined materials. 

177. An automated optical inspection device according to claim 172 and 
wherein said processing circuitry provides an output indication of the presence of at 
least three predetermined materials. 

178 - An automated optical inspection device according to any of claims 172 - 

177 and wherein said processing circuitry provides an output indication of the presence 
of copper. 



1 79. An automated optical inspection device according to any of claims 1 72 - 

1 78 and wherein said processing circuitry provides an output indication of the presence 
of gold. 
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180. An automated optical inspection device according to any of claims 172 - 

1 79 and wherein said processing circuitry provides an output indication of the presence 
of a solder mask material. 

5 181. An automated optical inspection device according to any of claims 1 72 - 

1 80 and wherein said processing circuitry provides an output indication of the presence 
of silver. 

1 82. An automated optical inspection device according to any of claims 172 - 

10 181 and wherein said processing circuitry provides an output indication of the presence 
of photoresist residue. 

183. An automated optical inspection device according to any of claims 172 - 

1 82 and wherein said processing circuitry provides an output indication of the presence 
15 of oxidized metal. 

184. An automated optical inspection device according to any of claims 172 - 

183 and wherein said image output represents a portion of the electrical circuit as 
viewed through a translucent overlay. 

20 

185. An automated optical inspection device suitable for inspection of a 
patterned article comprising: 

at least one detector providing an image output of at least a portion of a 
patterned article through a translucent overlay at least partially covering the patterned 
25 article; and 

processing circuitry receiving said image output and providing an output 
indication which is independent of the translucent overlay. 

186. An automated inspection device according to claim 185 and wherein the 
30 patterned article comprises an electrical circuit. 
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187. An automated inspection device according to claim 186 and wherein the 
electrical circuit comprises a printed circuit board. 

188. An automated inspection device according to claim 187 and wherein the 
printed circuit board comprises a ball grid array substrate. 

189. An automated inspection device according to claim 185 and wherein the 
processing circuitry is operative to provide an indication of parts of the pattern covered 
by the translucent overlay and parts of the pattern not covered by the translucent overlay. 

190. Apparatus for determining a location of a border in a color image, the 
image comprising at least two color populations, between a first color region associated 
with a first one of the two color populations and a second color region associated with a 
second one of the two color populations, both the first color region and the second color 
region being comprised in the color image, the apparatus comprising: 

a border identification unit operative to identify an approximate border 
location between said first color region and said second color region; 

a candidate border determination unit operative to determine a plurality 
of candidate border locations between the first color region and the second color region, 
each of said plurality of candidate border locations being determined by applying a 
corresponding border location method chosen from among a plurality of border location 
methods; 

a method selector operative to choose one method from among the 
plurality of border location methods as a preferred method; and 

a border determination unit operative to determine a location of a border 
between said first color region and said second color region by designating one of said 
plurality of candidate border locations associated with the preferred method as the 
border. 

191 • Apparatus for determining a location of a border in a color image, the 

image comprising at least two color populations, between a first color region associated 
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with a first one of the two color populations and a second color region associated with a 
second one of the two color populations, both the first color region and the second color 
region being comprised in the color image, the apparatus comprising: 

a border analysis map unit operative to provide a border analysis map 
5 comprising, for each distinct pair of color populations, an indication of a preferred 
method for identifying the location of a border between said color populations; 

a border identification unit operative to identify an approximate border 
location between said first color region and said second color region; and 

a border determination unit operative to determine a location of a border 
10 between said first color region and said second color region by using the preferred 
method indicated, in the border analysis map, for identifying the location of a border 
between said first color population and said second color population. 

1 92 - Apparatus for producing a color morphology map from a color image, the 

15 apparatus comprising: 

a reducing unit operative to reduce the color image to produce a reduced 
image comprising a plurality of color populations; 

a border determination unit operative to determine borders between 
adjacent ones of the plurality of color populations in the reduced image; 
20 a segmentation unit operative to segment the reduced image to produce a 

binary image comprising borders and at least one non-border region; 

a skeleton production unit operative to produce a skeleton image of the at 
least one non-border region, said skeleton image comprising a plurality of portions; and 
an assigning unit operative to assign to each portion of the skeleton a 
25 color identity corresponding to one of the color populations. 

Apparatus for identifying irregularities in a smooth curve in an image, 
the apparatus comprising: 

an identification unit operative to identify an ordered plurality of points 
30 along the smooth curve, each of said ordered plurality of points having at least one 
neighboring point in the ordered plurality of points; 
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a normal direction unit operative to determine a normal direction to the 
smooth curve at each of said ordered plurality of points; and 

an irregularity unit operative to identify a point of said ordered plurality 
of points as being part of an irregularity based, at least in part, on a local normal 
5 difference comprising a difference between the normal direction to the smooth curve at 
said point and a normal direction to the smooth curve at said at least one neighboring 
point of said point. 

194. Apparatus for classifying an irregularity in an image of an object, the 
10 apparatus comprising: 

a dimension determination unit operative to determine a feature depth 
and feature aperture for said irregularity; and 

a classification unit operative to classify said feature based, at least in 
part, on an output of said dimension determination unit. 

15 

195. Apparatus for identifying a defect in a portion of an image of an object, 
the portion being close to a location in the image identified as an edge, the apparatus 
comprising: 

an excluded direction unit operative to identify an excluded direction 
20 associated with the edge; and 

a defects unit operative to search for defects, close to the location 
identified as an edge, in a plurality of directions not including the excluded direction. 



196. Apparatus for identifying a surface defect in a region of an image of an 

25 object, the image comprising a multiplicity of pixels, the apparatus comprising: 

a subsampling unit operative to subsample the region in accordance with 
a subsampling ratio, thus producing a subsampled image of the region; 

a defect identification unit operative to identify a surface defect in the 
subsampled image; and 

30 a correspondence unit operative to identify a surface defect in the region 

corresponding to the surface defect in the subsampled image. 
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1 97. A method for inspection of patterned articles comprising: 

providing a polychromatic image output of at least a portion of a 
patterned article having a plurality of elements; and 

receiving said polychromatic image output and providing at least one of 
the following inspection functionalities: 

inspection of a metal coating on at least one of the elements in the 
plurality of elements; and 

inspection of an at least partially transparent coating on at least 
one element in the plurality of elements. 

1 98. A method for inspection of patterned articles comprising: 

providing a polychromatic image output of at least a portion of a 
patterned article comprising a plurality of elements; and 

receiving said polychromatic image output and providing the following 
inspection functionalities: 

inspection of a metal coating on at least one element in the 
plurality of elements; and 

inspection of an at least partially transparent coating on at least 
one element in the plurality of elements. 

1 99. A method for inspection of an article comprising: 

providing a polychromatic image output of at least a portion of an article; 

and 

receiving said polychromatic image output and providing edge detection 
to sub-pixel accuracy operative to distinguish borders among regions of different color. 

200. A method for inspection of an article comprising: 

providing an image output of at least a portion of an article having at 
least three different regions, each of which regions is distinguished at least by an optical 
characteristic; and 
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receiving said image output and providing edge detection to sub-pixel 
accuracy operative to distinguish borders of at least two of said at least three different 
regions. 

201 . A method for inspection of an article comprising: 
providing an image output of at least a portion of an article; and 
receiving said image output and providing mapping based at least 

partially on said image output which identifies materials in each of different regions of 
said at least a portion of said article having a common border. 

202. A method for inspection of an article comprising: 

providing a polychromatic image output of at least a portion of an article; 

and 

receiving said polychromatic image output and providing at least one of 
the following inspection functionalities: 

segmentation of said polychromatic image into a segmented color image 
comprising a plurality of color regions each having one of a plurality of representative 
color possibilities; 

binarization of the segmented color image into a binary image including 
regions and borders; 

morphological erosion of the regions to provide a skeleton representing 
the regions; and 

assignment of color regions to elements in the skeleton. 

203. A method for inspection of an article comprising: 

providing a polychromatic image output of at least a portion of an article; 

and 

receiving said polychromatic image output and providing a color image 
representing said article, the color image comprising a morphologically eroded 
polychromatic image of at least part of said article. 
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204- A method for inspection of an article comprising: 

providing a polychromatic image output of at least a portion of an article; 

and 

receiving said polychromatic image output and providing a color image 
representing said article, the color image comprising a map which indicates borders 
between different regions of an article and identifies materials in each of said different 
regions having a common border. 

205. A method for inspection of a patterned article comprising: 

providing an image output of at least a portion of a patterned article 

comprising at least three materials having detectable edges; and 

receiving said image output and providing an inspection functionality 

comprising binarization of the image output to provide a binary image including region 

image data and border image data. 

A method for inspection of a patterned article comprising: 

providing an image output of at least a portion of a patterned article 

comprising a plurality of materials having detectable borders between the plurality of 

materials; and 

receiving said image output and providing an inspection functionality 
comprising pattern analysis of image data relating to the borders to provide an indication 
of the presence of nicks and protrusions along said borders. 

207. A method for inspection of an article comprising: 

providing an image output of at least a portion of an article comprising 
multiple materials having detectable borders between the materials; 

receiving said image output and providing at least one of the following 
inspection functionalities: 

segmentation of said image output into a segmented image including 
region image data and border image data; 

analysis of the region image data using a first technique; and 
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analysis of the border image data using a second technique, different 
from the first technique. 

208. A method for inspection of an article comprising: 

5 providing a polychromatic image output of at least a portion of an article 

comprising multiple materials having detectable borders between the materials; and 

receiving said polychromatic image output and providing at least one of 
the following inspection functionalities: 

segmentation of said polychromatic image output into a segmented color 
1 0 image including color regions chosen from among a set of predetermined colors; 

analysis of the region image data relating to a first color using a first 

technique; and 

analysis of the region image data relating to a second color using a 
second technique, different from the first technique. 

15 

209. A method for inspection of an article comprising: 

providing an image output of at least a portion of an article comprising 
multiple materials having detectable borders between the materials; and 

receiving said image output and providing at least one of the following 
20 inspection functionalities: 

segmentation of said image output into a segmented image including 
region image data and border image data; and 

smoothing the region image data in a manner not influenced by at least 
some of the border image data. 

25 

210. A method for inspection of an electrical circuit comprising: 

providing an image output of at least a portion of an electrical circuit 
comprising multiple materials; and 

receiving said image output and providing an output indication of the 
30 presence of at least one predetermined material. 
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211. A method for inspection of a patterned article comprising: 

providing an image output of at least a portion of a patterned article 
through a translucent overlay at least partially covering the patterned article; and 

receiving said image output and providing an output indication which is 
5 independent of the translucent overlay. 

212 - A method for automatically optically inspecting color images of objects, 
comprising: 

acquiring polychromatic images of an object; 
10 processing the polychromatic images and outputting a report indicating 

candidate defects and features chosen from among a predetermined set of features, and 
providing an image of a region surrounding each said candidate defect and feature. 

213 - Apparatus for automatically optically inspecting color images of objects, 
1 5 comprising: 

a sensor operative to acquire polychromatic images of an object; 
an image processor operative to process the polychromatic images and 
output a report indicating candidate defects and features chosen from among a 
predetermined set of features, and to provide an image of a region surrounding each said 
20 candidate defect and feature. 

214 - Apparatus according to claim 213 and wherein the processor includes a 
binary image processor operative to process binary images generated from the 
polychromatic image, and a color image processor operative to process the 

25 polychromatic images. 

215 - Apparatus according to claim 213 and further comprising a 
morphological image processor operative to generate a skeleton map from a multicolor 
image derived from the polychromatic image. 
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Practitioner's Docket No. 

Optional Customer No. Bar Code 

lllllllllllUlllllllllllllll 

00140 

PATENT TRADEMARK OFFICE 



COMBINED DECLARATION AND POWER OF ATTORNEY 

(ORIGINAL, DESIGN, NATIONAL STAGE OF PCT, SUPPLEMENTAL, DIVISIONAL, 
CONTINUATION, OR C-I-P) 



As a below named inventor, I hereby declare that: 

TYPE OF DECLARATION 
This declaration is of the following type: 

(check one applicable item below) 

[ ] original. 
[ ] design. 

NOTE: With the exception of a supplemental oath or declaration submitted in a reissue, a supplemental oath or 

declaration is not treated as an amendment under 37 CFR 1.3 J 2 (Amendments after allowance). M.P.E.P. Section 
714.16, 7 ,h Ed. 

[ ] supplemental. 

NOTE: If the declaration is for an International Application being fled as a divisional, continuation or continuation-in- 
part application, do not check next item; check appropriate one of last three items. 

[x] national stage of PCT. 

NOTE: If one of the following 3 items apply, then complete and also attach ADDED PAGES FOR DIVISIONAL, 
CONTINUATION OR C-I-P. 

NOTE: See 37 C.F.R Section 1.63(d) (continued prosecution application) for use of a prior nonprovisional application 

declaration in the continuation or divisional application being filed on behalf of the same or fewer of the inventors 
named in the prior application. 

[ ] divisional. 
[ ] continuation. 

NOTE: Where an application discloses and claims subject matter not disclosed in the prior application, or a continuation 
or divisional application names an inventor not named in the prior application, a continuation-in-part application 
must be filed under 37 C.F.R. Section 1.53(b) (application filing requirements-nonprovisional application). 

[ ] continuation-in-part (C-I-P). 



PATENT 
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INVENTORSHIP IDENTIFICATION 



WARNING: If the inventors are each not the inventors of all the claims, an explanation of the facts, including the 

ownership of all the claims at the time the last claimed invention was made, should be submitted. 

My residence, post office address and citizenship are as stated below, next to my name. I believe that I 
am die original, first and sole inventor (if only one name is listed below) or an original, first and joint 
inventor (if plural names are listed below) of the subject matter that is claimed, and for which a patent 
is sought on the invention entitled: 

TITLE OF INVENTION 



OPTICAL INSPECTION SYSTEM 



SPECIFICATION IDENTIFICATION 

The specification of which: 

(complete (a), (b), or (c)) 

(a) [ ] is attached hereto. 

NOTE: "The following combinations of information supplied in an oath or declaration filed on the application filing date 
with a specification are acceptable as minimumsfor identifying a specification and compliance with any one of the 
items below will be accepted as complying with the identification requirement of 37 C.F.R. Section 1.63: 

"(I) name ofinventor(s), and reference to an attached specification which is both attached to the oath or 
declaration at the time of execution and submitted with the oath or declaration on filing; 

"(2) name of inventor (s), and attorney docket number which was on the specification as filed; or 

"(3) name of inventor (s), and title which was on the specification as filed. " 

Notice of My 13. 1995 (1177 O.G. 60). 

(b) [ J was filed on, , [ ] as Application No. 

[ ] and was amended on (if applicable). 

NOTE: Amendments filed after the original papers are deposited with the PTO that contain new matter are not accorded a 
filing date by being referred to in the declaration. Accordingly, the amendments involved are those filed with the 
application papers or, in the case of a supplemental declaration, are those amendments claiming matter not 
encompassed in the original statement of invention or claims. See 37 C.F.R. Section 1.67. 

NOTE: "The following combinations of information supplied in an oath or declaration filed after the filing date are 

acceptable as minimums for identifying a specification and compliance with any one of the items below will be 
accepted as complying with the identification requirement of 37 C.F.R. Section 1.63: 

(A) application number (consisting of the series code and the serial number, e.g., 08/123, 456); 

(B) serial number and filing date; 

(C) attorney docket number which was on the specification as filed; 

(D) title which was on the specification as filed and reference to an attached specification which is 
both attached to the oath or declaration at the time of execution and submitted with the oath or 
declaration; or 

(E) title which was on the specification as filed and accompanied by a cover letter accurately 
identifying the application for which it was intended by either the application number (consisting of the 
series code and the serial number, e.g., 08/123,456), or serial number and filing date. Absent any 
statements) to the contrary, it will be presumed that the application filed in the PTO is the application 
which the inventor(s) executed by signing the oath or declaration. 

M.P.E.P. Section 601.01(a), 7th ed. 
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(c) [X] was described and claimed in PCT International Application No. PCT/ 1 LOO/004 jfl ed 

on and as amended under PCT Article 19 on (if any) 

July 23, 2000 

SUPPLEMENTAL DECLARATION (37 C.F.R. Section 1.67(b)) 

(complete the following where a supplemental declaration is being submitted) 

[ ] I hereby declare that the subject matter of the 

[ ] attached amendment 

[ ] amendment filed on . 

was part of my/our invention and was invented before the filing date of the original 
application, above identified, for such invention. 



ACKNOWLEDGMENT OF REVIEW OF PAPERS AND DUTY OF CANDOR 

I hereby state that I have reviewed and understand the contents of the above- identified 
specification, including the claims, as amended by any amendment referred to above. 

I acknowledge the duty to disclose information, which is material to patentability as defined in 
37, Code of Federal Regulations, Section 1.56, 

(also check the following items, if desired) 

[ ] and which is material to the examination of this application, namely, information 
where there is a substantial likelihood that a reasonable Examiner would consider it 
important in deciding whether to allow the application to issue as a patent, and 

[ ] in compliance with this duty, there is attached an information disclosure 
statement, in accordance with 37 C.F.R. Section 1.98. 

PRIORITY CLAIM (35 U.S.C. Section 119(a)-(d)) 

NOTE: "The claim to priority need be in no special form and may be made by the attorney or agent if the foreign 

application is referred to in the oath or declaration as required by Section J. 63. The claim for priority and the 
certified copy of the foreign application specified in 35 U.S. C. Section J 19(b) must be filed in the case of an 
interference (Section 1.630), when necessary to overcome the date of a reference relied upon by the examiner, when 
specifically required by the examiner, and in all other situations, before the patent is granted. If the claim for 
priority or the certified copy of the foreign application is filed after the date the issue fee is paid, it must be 
accompanied by a petition requesting entry and by the fee set forth in Section 1. 1 7(i). If the certified copy is not in 
the English language, a translation need not be filed except in the case of interference; or when necessary to 
overcome the date of a reference relied upon by the examiner; or when specifically required by the examiner, in 
which event an English language translation must be filed together with a statement that the translation of the 
certified copy is accurate. "37 C.F.R. Section 1.55(a). 

I hereby claim foreign priority benefits under Title 35, United States Code, Section 119(a)-(d) 
of any foreign application(s) for patent or inventor's certificate or of any PCT international 
application(s) designating at least one country other than the United States of America listed below 
and have also identified below any foreign application(s) for patent or inventor's certificate or any 
PCT international application(s) designating at least one country other than the United States of 
America filed by me on the same subject matter having a filing date before that of the application(s) of 
which priority is claimed. 

(complete (d) or (e)) 
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(d) [ ] no such applications have been filed. 

(e) [X] such applications have been filed as follows. 

NOTE: Where item (c) is entered above and the International Application which designated the U.S. itself claimed priority 
check item (e), enter the details below and make the priority claim. 



PRIOR FOREIGN/PCT APPLICATION^) FILED WITHIN 12 MONTHS 
(6 MONTHS FOR DESIGN) PRIOR TO THIS APPLICATION 
AND ANY PRIORITY CLAIMS UNDER 35 U.S.C. SECTION 119(a)-(d) 



COUNTRY (OR 
INDICATE IF 
PCT) 


APPLICATION NUMBER 


DATE OF FILING 
DAY, MONTH, YEAR 


PRIORITY 
CLAIMED 
UNDER 35 USC 
119 


Israel 


131092 


25 July 1999 


]YES r 1NO 








[ ]YES r 1NO 








[ ]YES r 1NO 








[ ]YES r 1NO 








[ ]YES r 1NO 



CLAIM FOR BENEFIT OF PRIOR U.S. PROVISIONAL APPLICATION(S) 

(35 U.S.C. Section 119(e)) 

I hereby claim the benefit under Title 35, United States Code, Section 1 19(e) of any United 
States provisional application(s) listed below: 

PROVISIONAL APPLICATION NUMBER FILING DATE 

/ 

/ 

/ 

CLAIM FOR BENEFIT OF EARLIER U.S./PCT APPLICATION(S) 
UNDER 35 U.S.C. SECTION 120 

[ ] The claim for the benefit of any such applications are set forth in the attached 

ADDED PAGES TO COMBINED DECLARATION AND POWER OF ATTORNEY 
FOR DIVISIONAL, CONTINUATION OR CONTINUATION-IN-PART (C-I-P) 
APPLICATION. 
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ALL FOREIGN APPLICATION^), IF ANY, FILED MORE THAN 12 MONTHS 
(6 MONTHS FOR DESIGN) PRIOR TO THIS U.S. APPLICATION 



NOTE: If the application filed more than 12 months from the filing date of this application is a PCT filing forming the 

basis for this application entering the United States as (1) the national stage, or (2) a continuation, divisional or 
continuation-in-part, then also complete ADDED PAGES TO COMBINED DECLARATION AND POWER OF 
ATTORNEY FOR DIVISIONAL, CONTINUATION OR C-I-P APPLICATION for benefit of the prior U.S. or PCT 
application(s) under 35 U.S.C. Section 120. 



POWER OF ATTORNEY 

I hereby appoint the following practitioner(s) to prosecute this application and transact all 
business in the Patent and Trademark Office connected therewith. 

(list name and registration number) 

JOSEPH H. HANDELMA N, 26179 JULIAN H. CQHENJ20302 

JOHN RICHARD S, 3 1 053 WILLIAM R. EVANS 25858 

RICHARD J. STRE ET, 25765 JANET I. CO RD, 33778 

PETER D. GALLOWAY, 27885 CLIFFORD J. MASS, 30086 



IAIN C. BAILLIE L 24090 CYNTHIA R. MILLER, 34678 

RICHARD P. BERG, 28145 "~ " ~ 



(Check the following item, if applicable) 

[ ] I hereby appoint the practitioner(s) associated with the Customer Number provided 
below to prosecute this application and to transact all business in the Patent and 
Trademark Office connected therewith. 

[ ] Attached, as part of this declaration and power of attorney, is the authorization of the 
above-named practitioner(s) to accept and follow instructions from my 
representati ve(s) . 

NOTE: "Special care should be taken in continuation or divisional applications to ensure that any change of 

correspondence address in a prior application is reflected in the continuation or divisional application. For 
example, where a copy of the oath or declaration from the prior application is submitted for a continuation or 
divisional application filed under 37 CFR 1.53(b) and the copy of the oath or declaration from the prior 
application designates an old correspondence address, the Office may not recognize, in the continuation or 
divisional application, the change of correspondence address made during the prosecution of the prior application. 
Applicant is required to identify the change of correspondence address in the continuation or divisional application 
to ensure that communications from the Office are mailed to the current correspondence address. 37 CFR 
1.63(d)(4)." Section 601.03, M.P.E.P., 7th Ed 
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SEND CORRESPONDENCE TO DIRECT TELEPHONE CALLS TO: 

(Name and telephone number) 
Ladas & Parry Julian H. Cohen 

26 West 61 st Street (212) 708-1887 

New York, N.Y. 10023 



(complete the following if applicable) 

Since this filing is a [ ] continuation [ ] divisional there is attached hereto a Change of 
Correspondence Address so that there will be no question as to where the PTO should direct all 
correspondence . 



DECLARATION 

I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these statements 
were made with the knowledge that willful false statements and the like so made are punishable by 
fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code, and that 
such willful false statements may jeopardize the validity of the application or any patent issued 
thereon. 
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SIGNATURE(S) 

NOTE: Carefully indicate the family (or last) name, as it should appear on the filing receipt and all other document. 

NOTE: Each inventor must be identified by full name, including the family name, and at least one given name without 
abbreviation together with any other given name or initial, and by his/her residence, post office address and 
country of citizenship. 37 C.F.R. Section 1.63(a)(3). 

NOTE: Inventors may execute separate declarations/oaths provided each declaration/oath sets forth all the inventors. 

Section J. 63(a)(3) requires that a declaration/oath, inter alia, identify each inventor and prohibits the execution of 
separate declarations/ oaths which each sets forth only the name of the executing inventor. 62 Fed. Reg. 53, 131, 
53,142, October 10, 1997, 

Full name of sole or first inventor 

» JJri Gold 

s^"^ "(Given Name) (Middle Initial or Name) family ( Or Last Name) 

Inventor's signature (^) 

Date pju 3 / fr- r &Z Country of Citizenship _ 



Full name of second joint inventor, if any 

^ JLLL- Parente 

^ (Given Name) (MitMeJjaitial or Name) family (Ur Last Name) 




p^H.nPg JlshorMe Zion, Israel 



Post Office Address 19 Harashba Street, Rishon 1e Zion 75484, Israel 



Full name of third joint inventor, if any 

Tally Gal it-Be rn^ht.piri 

% ( Given Nam e) , JMiddle Initial or Name) family (Ur Last Name) 

*j) Inventor's signatu re (fi) > ,X^~~ — 

T>af«^ ) (o « TAm . J 001. Country of Citizenship ISRAEL 

,^,„^ lavne, Israel ZZX — 

7 Hanegev Street, Yavne 81514, Israel 
Post Office Address * — * 
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(check proper box(es) for any of the following added page(sj 
that form a part of this declaration) 

Signature for fourth and subsequent joint inventors. Number of pages added _ 



Signature by administrator(trix), executor(trix) or legal representative for deceased or 
incapacitated inventor. Number of pages added 



Signature for inventor who refuses to sign or cannot be reached by person authorized under 
37 C.F.R. Section 1.47. Number of pages added 



Added page for signature by one joint inventor on behalf of deceased inventor(s) where legal 
representative cannot be appointed in time. (37 C.F.R. Section 1.47) 



Added pages to combined declaration and power of attorney for divisional, continuation, or 
continuation-in-part (C-I-P) application. 

[ ] Number of pages added 



Authorization of practitioner(s) to accept and follow instructions from representative. 



(If no further pages form a part of this Declaration, 
then end this Declaration with this page and check the following item) 



I } This declaration ends with this page. 
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U ,1 ij V 0 £ 



■m mi i r 



Attorney's Docket No. 



ADDED PAGE TO COMBINED DECLARATION AND POWER OF 
ATTORNEY FOR SIGNATURE BY FOURTH AND SUBSEQUENT INVENTORS 

Full name of fourth joint Inventor, if any 

Edward Baranovskv 

(GIVEN NAME) 

Inventor's signatured 

Date \/3^ ftM 0^ WC/' Country of Citizenship _ 

nr-ir1rn-r X Rehovt ! Isr ^ '^^^ 
Post Office Address _ 




20 Hameltzer Street, Rehovot, Israel 



Full name of fifth joint inventor, if any 

Tami r Marcjalit 



(GKZH N*ME) (MIDDLE OR NAME) FAMILY (OR LAST NAME) 

Inventor's sjgna|tjre/_ ^ r ^^. y /\—- 



Inventor's signatures — <_ 

Date V 2 1 d. ' r^OQcP ^Onwntrv of Citizenship Israe1 

Jjazor. IsragT _T7~/>C 



Post Office Address . 



60 Mazor, D.N. Hamerkaz 73160, Israel 



Full name of sixth joint inventor, if any 



(GIVEN NAME) (MIDDLE INITIAL OR NAME) FAMILY (OR LAST NAME) 

Inventor's signature _ _____ — 

Date . Country of Citizenship 

Residence __ — — 



Post Office Address _ 



(Added Page to Combined Declaration and Power of Attorney for Signature by Fourth 
Subsequent Inventors [1-2]) 



